Scope of the Series

Mobile Robotics is a series devoted to applications of mobile robots and interaction of intelligent systems in different parts of the robot. The topics which this series is exploring are robot control, navigation system, intelligent systems, learning systems, deep learning systems, robot design, and applications in different environments. The series also welcomes research work in the area of mobile robotics development.

Contents

**Preface VII**

**Mobile Robots 23**

Ebrahim A. Mattar

Chapter 1 **A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions 1** Zool Hilmi Ismail and Nohaidda Sariff

Chapter 2 **Motion Control and Velocity-Based Dynamic Compensation for**

Chapter 3 **Theoretical and Experimental Collaborative Area Coverage**

Chapter 4 **Mobile Robot Feature-Based SLAM Behavior Learning, and**

Chapter 5 **Mobile Robot Navigation in Indoor Environments: Geometric,**

Ramón Barber, Jonathan Crespo, Clara Gómez, Alejandra C.

Cristiano Premebida, Rares Ambrus and Zoltan-Csaba Marton

Chapter 7 **Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout 129**

Chapter 8 **Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration 151** Tao Song, Fengfeng (Jeff) Xi and Shuai Guo

**Topological, and Semantic Navigation 87**

**Schemes Using Mobile Agents 43** Sotiris Papatheodorou and Anthony Tzes

**Navigation in Complex Spaces 67**

Hernámdez and Marina Galli

Chapter 6 **Intelligent Robotic Perception Systems 111**

Mohammed A. H. Ali and Musa Mailah

Felipe Nascimento Martins and Alexandre Santos Brandão

## Contents

#### **Preface XI**


Tao Song, Fengfeng (Jeff) Xi and Shuai Guo

#### Chapter 9 **4WD Robot Posture Estimation by Radial Multi-View Visual Odometry 177**

Edgar Alonso Martínez-García and Luz Abril Torres-Méndez

Preface

interest for each of our readers.

This book contains a selection of research work that focuses on the development of mobile robots. In this way we can discover the evolution of multi-agent robots in recent years. The investigations present alternatives to various problems such as a dynamic model based on speed for mobile robots with a differential drive, collaborative control schemes of a system for mobile ground robots, learning systems applied to mobile robots and the behavior of naviga‐ tion related to interior environments, and the application of artificial intelligence algorithms for learning sensory data based on learned models. This book also presents a road mapping system and the extraction of features for navigation with mobile robots in roundabout envi‐ ronments and road monitoring, tracking routes of a mobile wheeled manipulator designed for manufacturing processes, systems where a Kalman filter estimator is extended for visual od‐ ometry, and finally the design of an on-chip system for the execution of cognitive agents.

We hope that the present research work will enrich and contribute to ideas and elements of

I appreciate the professional work of each of the researchers who have contributed by shar‐ ing their work, developments, and ideas, which allowed us to present this book and share

Engineering Faculty of the Autonomous University of Queretaro, Mexico

**Efren Gorrostieta Hurtado, Dr Eng**

all this information with readers and the rest of the scientific community.

#### Chapter 10 **IntelliSoC: A System Level Design and Conception of a Systemon-a-Chip (SoC) to Cognitive Agents Architecture 199** Diego Ferreira, Augusto Loureiro da Costa and Wagner Luiz Alves De Oliveira

## Preface

Chapter 9 **4WD Robot Posture Estimation by Radial Multi-View Visual**

Chapter 10 **IntelliSoC: A System Level Design and Conception of a Systemon-a-Chip (SoC) to Cognitive Agents Architecture 199** Diego Ferreira, Augusto Loureiro da Costa and Wagner Luiz Alves

Edgar Alonso Martínez-García and Luz Abril Torres-Méndez

**Odometry 177**

**VI** Contents

De Oliveira

This book contains a selection of research work that focuses on the development of mobile robots. In this way we can discover the evolution of multi-agent robots in recent years. The investigations present alternatives to various problems such as a dynamic model based on speed for mobile robots with a differential drive, collaborative control schemes of a system for mobile ground robots, learning systems applied to mobile robots and the behavior of naviga‐ tion related to interior environments, and the application of artificial intelligence algorithms for learning sensory data based on learned models. This book also presents a road mapping system and the extraction of features for navigation with mobile robots in roundabout envi‐ ronments and road monitoring, tracking routes of a mobile wheeled manipulator designed for manufacturing processes, systems where a Kalman filter estimator is extended for visual od‐ ometry, and finally the design of an on-chip system for the execution of cognitive agents.

We hope that the present research work will enrich and contribute to ideas and elements of interest for each of our readers.

I appreciate the professional work of each of the researchers who have contributed by shar‐ ing their work, developments, and ideas, which allowed us to present this book and share all this information with readers and the rest of the scientific community.

#### **Efren Gorrostieta Hurtado, Dr Eng**

Engineering Faculty of the Autonomous University of Queretaro, Mexico

**Chapter 1**

**Provisional chapter**

**A Survey and Analysis of Cooperative Multi-Agent**

**A Survey and Analysis of Cooperative Multi-Agent** 

Research in the area of cooperative multi-agent robot systems has received wide attention among researchers in recent years. The main concern is to find the effective coordination among autonomous agents to perform the task in order to achieve a high quality of overall performance. Therefore, this paper reviewed various selected literatures primarily from recent conference proceedings and journals related to cooperation and coordination of multi-agent robot systems (MARS). The problems, issues, and directions of MARS research have been investigated in the literature reviews. Three main elements of MARS which are the type of agents, control architectures, and communications were discussed thoroughly in the beginning of this paper. A series of problems together with the issues were analyzed and reviewed, which included centralized and decentralized control, consensus, containment, formation, task allocation, intelligences, optimization and communications of multi-agent robots. Since the research in the field of multi-agent robot research is expanding, some issues and future challenges in MARS are recalled, discussed and clarified with future directions. Finally, the paper is concluded with some

**Keywords:** cooperative mobile robots, multi-agent robot systems, coordination, control,

Research on the multi-agent robot systems has been conducted since late 80s as it provides a more efficient and robust system compared to a single robot. ALLIANCE [1] and ACTRESS [2] robot are among of the earliest heterogeneous multi-agent robots developed by previous

DOI: 10.5772/intechopen.79337

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

**Robot Systems: Challenges and Directions**

**Robot Systems: Challenges and Directions**

Zool Hilmi Ismail and Nohaidda Sariff

Zool Hilmi Ismail and Nohaidda Sariff

http://dx.doi.org/10.5772/intechopen.79337

**Abstract**

communication

**1. Introduction**

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

recommendations with respect to multi-agent systems.

#### **A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions**

DOI: 10.5772/intechopen.79337

Zool Hilmi Ismail and Nohaidda Sariff Zool Hilmi Ismail and Nohaidda Sariff

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79337

#### **Abstract**

Research in the area of cooperative multi-agent robot systems has received wide attention among researchers in recent years. The main concern is to find the effective coordination among autonomous agents to perform the task in order to achieve a high quality of overall performance. Therefore, this paper reviewed various selected literatures primarily from recent conference proceedings and journals related to cooperation and coordination of multi-agent robot systems (MARS). The problems, issues, and directions of MARS research have been investigated in the literature reviews. Three main elements of MARS which are the type of agents, control architectures, and communications were discussed thoroughly in the beginning of this paper. A series of problems together with the issues were analyzed and reviewed, which included centralized and decentralized control, consensus, containment, formation, task allocation, intelligences, optimization and communications of multi-agent robots. Since the research in the field of multi-agent robot research is expanding, some issues and future challenges in MARS are recalled, discussed and clarified with future directions. Finally, the paper is concluded with some recommendations with respect to multi-agent systems.

**Keywords:** cooperative mobile robots, multi-agent robot systems, coordination, control, communication

#### **1. Introduction**

Research on the multi-agent robot systems has been conducted since late 80s as it provides a more efficient and robust system compared to a single robot. ALLIANCE [1] and ACTRESS [2] robot are among of the earliest heterogeneous multi-agent robots developed by previous

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### 2 Applications of Mobile Robots

researchers. The benefits received from information sharing among agents, data fusion, distribution of task, time and energy consumption have made the multi-agents research still relevant until present.

the agents. Therefore, this paper thoroughly explains each of the key elements with related examples from previous research and followed by the issues and directions of the multi-agent

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

http://dx.doi.org/10.5772/intechopen.79337

3

Multi-agent robots can be divided into two categories which are homogeneous and heterogeneous. The agents become homogeneous when the physical structures or capabilities of the agents/individuals are identical (**Figure 1**). The capabilities for heterogeneous agents are not identical and they are different among robots, where each robot has its own specialization or specific task to complete [8]. Besides that, the physical structures of heterogeneous agents are

Research carried out by Sugawara and Sano [9] and Hackwood and Beni [10] have proven that their homogeneous agents that have identical structures and identical capabilities can perform the task efficiently. However, for Li and Li [11], the heterogeneous agents are more applicable than homogeneous agents in the real world. Therefore, instead of focusing on homogeneous agents, current researchers are also concerned about heterogeneous agent's issues [1–6, 11–15]. The agent's physical structures and capabilities which are not identical

There are two researchers known as Parker [18] and Goldberg [19] who compared the task coverage and interference between homogeneous and heterogeneous agents. Parker discovered that the task coverage for homogeneous agents is maximum compared to heterogeneous. This is because the homogeneous agents execute the same task at one time, while the heterogeneous agents need to distribute their task to another agent during the execution. Due to the task distributions among heterogeneous agents, the interference becomes higher compared to homogeneous agents, as proven in Goldberg's research [19]. As a result, we can summarize that the selection of a homogeneous or heterogeneous agent depends on the research application. Since the capability of heterogeneous agents is not identical, it becomes a challenging issue especially in finding consensus among agent during execution of the task. **Table 1** shows the research conducted by previous researchers using their heterogeneous agents.

have made the agents fall into these heterogeneous agents categories [16, 17].

**2.1. Types of agents: homogeneous and heterogeneous**

also not identical among them (**Figures 2** and **3**).

**Figure 1.** Multi-agent (homogeneous agents) control with broadcast [33].

robot systems.

There were many researchers who focused on cooperative multi-agent research. The most challenging part was to provide a robust and intelligent control system so that the agents can communicate and coordinate among them to complete the task. Hence, it has been found that designing the control architecture, communication, and planning system were the major issues discussed and solved among researchers. Other than that, improvement to the existing coordination techniques, optimal control architectures, and communication were also the main highlights in the previous research. A few examples of cooperative multi-agent robots applications are soccer robot [3], unmanned guided vehicles (UGV's) and unmanned aerial vehicles (UAV's) [4], micro chain [5], and paralyzed robot [6].

There were two main reviewed papers proposed by Cao and Zhi Yan which were related to cooperative multi-agent research. Cao et al. [7] proposed a paper that represents the antecedents and direction of the cooperative mobile robot in the mid-1990s (most of the reviewed papers were published from 1990 to 1995). There were several issues discussed such as group architecture, resource conflict, the origin of cooperation, learning, and geometric problem. The applications and critical survey of the issues and direction of cooperative robots based on existing motivation have been indicated. Besides that, there were also a survey and an analysis of multi-robot coordination proposed by Yan et al. [8] in 2013 (most of the reviewed papers were published from 2000 to 2013). They presented a systematic survey and analysis of multiple mobile robot systems coordination. Related problems such as communication mechanism, a planning strategy, and a decision-making structure have been reviewed. In addition, various additional issues of cooperative MARS have been highlighted in these reviewed papers. Most of the papers were published from 2010 to 2015 which the recent research papers on cooperative multi-agent systems have been reviewed.

The main contributions of this paper are (i) the most reflected and affected key elements and current issues in cooperative mobile robots and (ii) directions and future challenges for the multi-agents robot, with recommendations and related suggestions. The remain sections of the paper are structured as follows: the first section discusses three main categories of multiagent robot systems, the second section focuses on discussion of problems and some current issues of multi-agent systems and the final section is the conclusions with some challenges and recommendations for future research direction in the field of cooperative multi-agent systems.

#### **2. Key elements of cooperative multi-agent robot systems**

A wide means of research in cooperative multi-agent robots systems have focused on the three main elements which are (1) types of agents; homogeneous and heterogeneous, (2) control architectures; reactive, deliberative and hybrid, and (3) communication; implicit and explicit. In order to provide efficient coordination among multi-agent robots, the selections and designs of the control architecture and communication must possess a coherent behavior with the agents. Therefore, this paper thoroughly explains each of the key elements with related examples from previous research and followed by the issues and directions of the multi-agent robot systems.

#### **2.1. Types of agents: homogeneous and heterogeneous**

researchers. The benefits received from information sharing among agents, data fusion, distribution of task, time and energy consumption have made the multi-agents research still

There were many researchers who focused on cooperative multi-agent research. The most challenging part was to provide a robust and intelligent control system so that the agents can communicate and coordinate among them to complete the task. Hence, it has been found that designing the control architecture, communication, and planning system were the major issues discussed and solved among researchers. Other than that, improvement to the existing coordination techniques, optimal control architectures, and communication were also the main highlights in the previous research. A few examples of cooperative multi-agent robots applications are soccer robot [3], unmanned guided vehicles (UGV's) and unmanned aerial

There were two main reviewed papers proposed by Cao and Zhi Yan which were related to cooperative multi-agent research. Cao et al. [7] proposed a paper that represents the antecedents and direction of the cooperative mobile robot in the mid-1990s (most of the reviewed papers were published from 1990 to 1995). There were several issues discussed such as group architecture, resource conflict, the origin of cooperation, learning, and geometric problem. The applications and critical survey of the issues and direction of cooperative robots based on existing motivation have been indicated. Besides that, there were also a survey and an analysis of multi-robot coordination proposed by Yan et al. [8] in 2013 (most of the reviewed papers were published from 2000 to 2013). They presented a systematic survey and analysis of multiple mobile robot systems coordination. Related problems such as communication mechanism, a planning strategy, and a decision-making structure have been reviewed. In addition, various additional issues of cooperative MARS have been highlighted in these reviewed papers. Most of the papers were published from 2010 to 2015 which the recent research papers

The main contributions of this paper are (i) the most reflected and affected key elements and current issues in cooperative mobile robots and (ii) directions and future challenges for the multi-agents robot, with recommendations and related suggestions. The remain sections of the paper are structured as follows: the first section discusses three main categories of multiagent robot systems, the second section focuses on discussion of problems and some current issues of multi-agent systems and the final section is the conclusions with some challenges and recommendations for future research direction in the field of cooperative multi-agent

A wide means of research in cooperative multi-agent robots systems have focused on the three main elements which are (1) types of agents; homogeneous and heterogeneous, (2) control architectures; reactive, deliberative and hybrid, and (3) communication; implicit and explicit. In order to provide efficient coordination among multi-agent robots, the selections and designs of the control architecture and communication must possess a coherent behavior with

vehicles (UAV's) [4], micro chain [5], and paralyzed robot [6].

on cooperative multi-agent systems have been reviewed.

**2. Key elements of cooperative multi-agent robot systems**

relevant until present.

2 Applications of Mobile Robots

systems.

Multi-agent robots can be divided into two categories which are homogeneous and heterogeneous. The agents become homogeneous when the physical structures or capabilities of the agents/individuals are identical (**Figure 1**). The capabilities for heterogeneous agents are not identical and they are different among robots, where each robot has its own specialization or specific task to complete [8]. Besides that, the physical structures of heterogeneous agents are also not identical among them (**Figures 2** and **3**).

Research carried out by Sugawara and Sano [9] and Hackwood and Beni [10] have proven that their homogeneous agents that have identical structures and identical capabilities can perform the task efficiently. However, for Li and Li [11], the heterogeneous agents are more applicable than homogeneous agents in the real world. Therefore, instead of focusing on homogeneous agents, current researchers are also concerned about heterogeneous agent's issues [1–6, 11–15]. The agent's physical structures and capabilities which are not identical have made the agents fall into these heterogeneous agents categories [16, 17].

There are two researchers known as Parker [18] and Goldberg [19] who compared the task coverage and interference between homogeneous and heterogeneous agents. Parker discovered that the task coverage for homogeneous agents is maximum compared to heterogeneous. This is because the homogeneous agents execute the same task at one time, while the heterogeneous agents need to distribute their task to another agent during the execution. Due to the task distributions among heterogeneous agents, the interference becomes higher compared to homogeneous agents, as proven in Goldberg's research [19]. As a result, we can summarize that the selection of a homogeneous or heterogeneous agent depends on the research application. Since the capability of heterogeneous agents is not identical, it becomes a challenging issue especially in finding consensus among agent during execution of the task. **Table 1** shows the research conducted by previous researchers using their heterogeneous agents.

**Figure 1.** Multi-agent (homogeneous agents) control with broadcast [33].

#### 4 Applications of Mobile Robots

**Figure 2.** Examples of Heterogeneous agent (chained micro robots) [5].

**Figure 3.** The heterogeneous team includes a single UAV (Pelican Quadrotor) controlling many UGV's (Khepera II robots) [4].

#### **2.2. Control architectures: reactive, deliberative, and hybrid**

The selection of control architectures for multi-agent robots is based on the capabilities of each agent to work in the groups and it also depends on how the overall systems work. The control architectures can be classified into three categories which are (i) reactive, (ii) deliberative and (iii) hybrid (reactive and deliberative).

Reactive control is also known as decentralized control. Reactive control relies on the concept of perception-reaction where the agents will cooperate between agents based on direct perception, signal broadcast or indirect communication via environmental changes. It does not require a high-level of communication to interact with agents. There are a few approaches which are related to reactive control for multi-agent robots. Glorennec [20] coordinated multiagent robots by using fuzzy logic techniques to avoid obstacles and robots in the environment, whereas, research done by Lope et al. [12] coordinated their multi-agent robots by using the reinforcement learning algorithm based on the learning automata and ant colony optimization theory. Their multi-agent robots can organize the task by themselves to choose any task to be executed. It was proven that without interference from the central controller, the robots are capable of selecting their own task independently.

the leader and follower to a specific path. Finally, the decentralized control for stabilizing nonlinear multi-agent systems by using neural inverse optimal paper is carried out by Franco

**Robot task Type of robots Reason of heterogeneous**

striker, and defender

One paralyzed robot with multiple numbers of pusher robots

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

Group of agents (robots) acts as a goal keeper, middle field player,

Single UAV flies to control and allocate several UGV'S

Consists of leader and few

(active and passive) such as rotation, support, extension and

human operator, image processor, and global environment manager

Small to a medium size of heterogeneous teams

helicoidally modules

followers

Coordination of the micro robots chain [5] Consists of different modules

ACTRESS robot pushes the objects [2] 3 different robotors act as interface

Different robot, different task

http://dx.doi.org/10.5772/intechopen.79337

5

Same robot, different task for each group of agents since each agent has different capabilities and

Different robot, different task

Same agent, different state dimension among the leader and follower (not identical)

Different modules, different task/

Different agent, different task

Different agent, different task

characteristics

function

Multiple agents Same agent, different task

The pusher robots work among them and push the paralyzed robot to a certain point. The paralyzed robot is driven by

The ROBOCUP robot team plays a soccer

Unmanned aerial vehicles (UAV) acts as a supervisor to control unmanned ground vehicles (UGV) robots from any danger

Coordination of heterogeneous multiagents systems. Second order dynamics is the state of the leader while first order dynamics is the state of the followers [11]

Multi-agent robots construct four

ALLIANCE robot executes few tasks. The tasks are box pushing, puck gathering, marching, information, marching, hazardous and waste cleanup [1]

**Table 1.** Examples of heterogeneous agent's research.

different blocks [12]

and collide with obstacles [4]

the global system [6]

ball [3]

Deliberative approach relied on the high-level communication, rich sensor and complete representation of the environment which allow the planning action. This approach is also known as centralized approach. The input data (usually from the static environment) that represents the global map can be planned to drive the agents efficiently to the target point [6, 24, 25]. The hybrid approach represents the integration control between reactive and deliberative control. Both controls complement each other to find the robust control system in controlling multi-agents robot. In deliberative control, all of the planning processes are involved with the calculation of a global target. As for reactive control, it is more towards a local plan for the robot to avoid the obstacles. There are examples of hybrid approaches related to multi-agents

et al. [23].

research studies as shown in **Table 2** [5, 6, 24, 25].

Despite these approaches, Chen and Sun [21] proposed a new optimal control law as distributed control for multi-agent robots in finding consensus to avoid obstacles in the environment. Local information from neighbors is required in this research. It is proven that this approach is capable of solving consensus problem under obstacle avoidance scenarios. In terms of local information context, Vatankhah et al. [22] developed a unique adaptive controller to move


**Table 1.** Examples of heterogeneous agent's research.

**2.2. Control architectures: reactive, deliberative, and hybrid**

**Figure 2.** Examples of Heterogeneous agent (chained micro robots) [5].

are capable of selecting their own task independently.

(iii) hybrid (reactive and deliberative).

robots) [4].

4 Applications of Mobile Robots

The selection of control architectures for multi-agent robots is based on the capabilities of each agent to work in the groups and it also depends on how the overall systems work. The control architectures can be classified into three categories which are (i) reactive, (ii) deliberative and

**Figure 3.** The heterogeneous team includes a single UAV (Pelican Quadrotor) controlling many UGV's (Khepera II

Reactive control is also known as decentralized control. Reactive control relies on the concept of perception-reaction where the agents will cooperate between agents based on direct perception, signal broadcast or indirect communication via environmental changes. It does not require a high-level of communication to interact with agents. There are a few approaches which are related to reactive control for multi-agent robots. Glorennec [20] coordinated multiagent robots by using fuzzy logic techniques to avoid obstacles and robots in the environment, whereas, research done by Lope et al. [12] coordinated their multi-agent robots by using the reinforcement learning algorithm based on the learning automata and ant colony optimization theory. Their multi-agent robots can organize the task by themselves to choose any task to be executed. It was proven that without interference from the central controller, the robots

Despite these approaches, Chen and Sun [21] proposed a new optimal control law as distributed control for multi-agent robots in finding consensus to avoid obstacles in the environment. Local information from neighbors is required in this research. It is proven that this approach is capable of solving consensus problem under obstacle avoidance scenarios. In terms of local information context, Vatankhah et al. [22] developed a unique adaptive controller to move the leader and follower to a specific path. Finally, the decentralized control for stabilizing nonlinear multi-agent systems by using neural inverse optimal paper is carried out by Franco et al. [23].

Deliberative approach relied on the high-level communication, rich sensor and complete representation of the environment which allow the planning action. This approach is also known as centralized approach. The input data (usually from the static environment) that represents the global map can be planned to drive the agents efficiently to the target point [6, 24, 25]. The hybrid approach represents the integration control between reactive and deliberative control. Both controls complement each other to find the robust control system in controlling multi-agents robot. In deliberative control, all of the planning processes are involved with the calculation of a global target. As for reactive control, it is more towards a local plan for the robot to avoid the obstacles. There are examples of hybrid approaches related to multi-agents research studies as shown in **Table 2** [5, 6, 24, 25].


**Table 2.** Hybrid control architectures of multi-agent robots.

Every researcher has used different types of control architecture that are suitable for their system. They have come out with their own idea about the control architectures. Based on [26], hybrid architectures offer the most widespread solution in controlling intelligent mobile robots. Besides that, in a real world, agents also require acting in a dynamic and uncertain environment [6]. Subsequently, the hybrid approach allows the robot to navigate the target as well as avoiding the obstacles successfully within that environment [24].

by embedding a different kind of sensors among them. They will react to avoid obstacles among themselves if they sense signals from other agents [4, 10, 30–32]. However, due to limitation of hardware parts, the interaction via sensing has been replaced by using a radio or

robot (1 robot) UAV allocate UGV's [4] UAV equipped with a video camera with onboard Inertial

finder sensor

sensors

Real mobile robots equipped with CCD cameras

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

4 robots are equipped with side sensors. Different signals are emitted to differentiate between the robots (3 robots) and target

Measurement Unit (IMU). UGV equipped with onboard laser range

http://dx.doi.org/10.5772/intechopen.79337

7

Robots are equipped with a pair of photo sensors and a pair of IR

Agent to agent

local

global

Agent to agent

One to many agents (modules) for global. Agent to agents for

One way communication. One to all agents (broadcast) for

One way communication. One to all agents (broadcast)

4 robots equipped with ultrasound sensors

Explicit communication refers to the direct exchange of information between agents or via broadcast messages. This often requires onboard communication modules. Issues on designing the network topologies and communication protocol arise because these types of communication are similar to the communication network [3, 5–6, 33–39]. **Table 4** shows an example

> Robots equipped with communicating devices (off-the-shelf) radio modems and wireless Ethernet cards. Communication is based on underlying IP protocol either TCP-IP or UDP-IP

PC will send messages to the Mindstorm robot (paralyzed) to control Mirosot robots (pusher) by using infrared serial communications interface/

The broadcast signal sent from computer to all

Communication-based on conceptual mechanism of "sign-board" being used in inter-robot system

Command exchange protocol is used for communication between modules and PC by sending a message. The name of the protocol is

of explicit communications being used in the robot systems.

*I*<sup>2</sup> *C* protocol

transceiver

agents via Bluetooth

**Task Communication devices/network Interaction**

infrared communication.

Coordination of the ROBOCUP teams (middle size league) [3]

Developing the control architectures for chain micro

The pusher robots work cooperatively to push the paralyzed robot to a specific

Broadcast control framework for multi-agent coordination

Sign board based inter-robot communication in distributes

robotic system [34]

robots [5]

point [6]

[33]

**Task Sensors**

Multi-robots work together to avoid other robots, remove obstacles and pass objects [30]

Follower robots follow the leader while

Robot teams will track the target and push the

Multi-robot cooperatively collects the pucks in

**Table 3.** Implicit communication researches.

avoiding the obstacles [31]

box cooperatively [32]

the field [9]

The researchers who have focused on reactive architectures or known as decentralized approach [27] have claimed that decentralization will provide flexibility and robustness. However, Franco et al. [23] have different views where they agreed with the deliberative approach (centralized) is obviously good for their system although it is hard to control in a complex and large system due to technical and economic reasons. Sometimes, centralized control design totally depends on the system structure and it cannot handle structural changes. Once removed, it needs to be designed all over again. It is also costly and complex in terms of online computation and its control design.

#### **2.3. Communications: implicit and explicit**

Cooperation is usually based on some forms of communication. Communication is a mode of interactions between multi-agent robots. With an efficient communication system, the robot is capable of interacting, sharing and exchanging information. Communication also determines the success in mobile robots cooperation [28, 29]. Based on research by Cao et al. [7], there are three types of communication structures which are (i) interaction via the environment, (ii) interaction via sensing, and (iii) interaction via communications. However, this section will only focus on the two main types of interaction (ii) and (iii) which are important in the communication of mobile robots.

Implicit communication or also known as interaction via sensing refers to the local interactions between agents (agent to agent) as shown in **Table 3**. The agents will sense other agents


**Table 3.** Implicit communication researches.

Every researcher has used different types of control architecture that are suitable for their system. They have come out with their own idea about the control architectures. Based on [26], hybrid architectures offer the most widespread solution in controlling intelligent mobile robots. Besides that, in a real world, agents also require acting in a dynamic and uncertain environment [6]. Subsequently, the hybrid approach allows the robot to navigate the target as

High layer for central control Low-level embedded

**Task Deliberative (D) Reactive (R) Communication of D** 

Emit an attractive signal to move paralyzed robot to a specific target and to recruit another pusher robot to push the paralyzed robots (broadcast simple signal). It has a vision of the environment to determine the path

Introduce supervisor that assists a group of agents with centralized coverage control law and global trajectory tracking control law

**and R**

time

protocol

D broadcast emitted signal to R controller

Using a control law. Each law is active at a given

D and R communicate using command exchange

A force field approach used to define the pushers robots motion

Introduce control laws for coverage agents to avoid a collision and maintain proximity to a supervisor

layer based on behavior function

The researchers who have focused on reactive architectures or known as decentralized approach [27] have claimed that decentralization will provide flexibility and robustness. However, Franco et al. [23] have different views where they agreed with the deliberative approach (centralized) is obviously good for their system although it is hard to control in a complex and large system due to technical and economic reasons. Sometimes, centralized control design totally depends on the system structure and it cannot handle structural changes. Once removed, it needs to be designed all over again. It is also costly and complex in

Cooperation is usually based on some forms of communication. Communication is a mode of interactions between multi-agent robots. With an efficient communication system, the robot is capable of interacting, sharing and exchanging information. Communication also determines the success in mobile robots cooperation [28, 29]. Based on research by Cao et al. [7], there are three types of communication structures which are (i) interaction via the environment, (ii) interaction via sensing, and (iii) interaction via communications. However, this section will only focus on the two main types of interaction (ii) and (iii) which are important in the

Implicit communication or also known as interaction via sensing refers to the local interactions between agents (agent to agent) as shown in **Table 3**. The agents will sense other agents

well as avoiding the obstacles successfully within that environment [24].

terms of online computation and its control design.

**2.3. Communications: implicit and explicit**

**Table 2.** Hybrid control architectures of multi-agent robots.

Pusher robots push the paralyzed robot to a specified target point [6]

6 Applications of Mobile Robots

Solving dynamic problem for multi-agents by proposing a novel control scheme [25]

Movement of chain micro-robots [5]

communication of mobile robots.

by embedding a different kind of sensors among them. They will react to avoid obstacles among themselves if they sense signals from other agents [4, 10, 30–32]. However, due to limitation of hardware parts, the interaction via sensing has been replaced by using a radio or infrared communication.

Explicit communication refers to the direct exchange of information between agents or via broadcast messages. This often requires onboard communication modules. Issues on designing the network topologies and communication protocol arise because these types of communication are similar to the communication network [3, 5–6, 33–39]. **Table 4** shows an example of explicit communications being used in the robot systems.



the system. Effect of this high computation, the time as well as the energy consumption will be effected at some point. Therefore, to solve this problem, hybrid control approach [5, 6, 24, 25] has been proposed with objective to balance between centralized control and distributed control [23, 26, 48–52]. Besides that, alternative towards optimizing or minimizing the trajectory length, time and energy consumption [24] as well as adding the intelligences [11, 20, 22, 27, 31, 32, 53] has taken into consideration to reduce the computation time. In terms of scalability, adaptability and flexibility of the controller can be claimed lesser as compared to distributed control. Any changes especially dealing with dynamics will cause the repetition in the computing and sometimes will effect overall of the system with only a limited number of controllers. Thus,

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

http://dx.doi.org/10.5772/intechopen.79337

9

Distributed control had proven scalable, adaptive, flexible and robust for multi agents system not only in static but also in a dynamic environment [54]. Many researchers had proven that their distributed controller can work efficiently for their multi agent robot systems [12, 26, 31, 32, 34, 48–50, 53, 55–58]. Innocenti et al. [27] have proven that their ActivMedia Pioneer 2DX mobile robots can reach its target by using their fuzzy logic controller. Same goes to Chen and Sun [21], Vatankhah et al. [22] and Glorennec [20] where they develop the distributed controller purposely for obstacles avoidance for their multi agents by using a fuzzy, neuro fuzzy, and

In distributed, the main issue is the task has to be distributed in a robust an efficient manner to ensure that every agent is able to perform its individual task cooperatively with another agents to achieve certain target. Distributing task among heterogeneous agents [11, 15] is more crucial and complex comparing with homogeneous agents which are identical [20–22, 59]. Limited sensing range and low bandwidth are also among physical constraints in distributed approach. With a limited local information, the agent cannot predict and cannot control the group behavior effectively in some sense. Another issues in distributed such as consensus, formation, containment, task allocation, optimization and intelligence will also discussed

Since multi-agent robots need to interact and communicate together to work cooperatively, issue on finding consensus for the homogeneous and heterogeneous robot has attracted researchers' attention over the past few years. Consensus refers to the degree of agreement among multi-agents to reach certain quantities of interest. The main problem of consensus control in multi-agent robots is to design a distributed protocol by using local information which can guarantee the agreements between robots to reach certain tasks or certain states. Therefore, a large number of interest concerning on developing the consensus control distributed protocol for homogeneous and heterogeneous robots which can be classified into a leader following consensus [60] and leaderless consensus [61–66], (to name a few), have been

Each of heterogeneity agents is not identical and the states between agents are different which will cause difficulties in finding consensus. This is known as cooperative output consensus problem. This is a challenging issue for heterogeneous robots and there are a number of researchers who focused on the leaderless output consensus problem [13, 15, 68] and leaderfollower output consensus problem [13, 15, 69–71]. Wieland et al. [68] proposed an internal

centralized control sometimes does not fit with the dynamic environment.

a new optimal control protocol.

thoroughly in below section.

intensively studied by researchers recently [22, 67].

**3.2. Consensus**

**Table 4.** Explicit communication researches.

#### **3. Problems and issues of cooperative multi-agent robot systems**

Although researchers in recent years have addressed the issues of multi-agent robot systems (MARS), the current robot technology is still far from achieving many real world applications. Some real world MARS applications can be found in unmanned aerial vehicles (UAV's), unmanned ground vehicles (UGV's), unmanned underwater vehicles (UUV's), multi-robot surveillance, planetary exploration, search and rescue missions, service robots in smart homes and offices, warehouse management, as well as transportation. Therefore, in this paper, problems and issues related to cooperative multi-agent systems are discussed to improve the current approaches and to further expand the applications of MARS.

#### **3.1. Centralized and distributed control**

Based on Section 2.2, the differences between two types of control approaches have been highlighted. However, some problems and issues of both control system in coordinating multi agents will be discussed. By having centralized control, the global information of the environment has been used to calculate the path, trajectory or position of the agents before all [5, 6, 24–26, 40]. The information then can be sent directly to the agents by using a suitable communication medium. This is one advantage of this control where the agents can obtained the information directly from its central. Research by Azuma [33] shows that the central will sent the updated location directly to the agents by using a WIFI continuously until the agents reach the target point. The quadratic equation is used to calculate agent performances while Simultaneous Perturbation Stochastic Approximation is the algorithm used for the control design [41]. Besides that, A\* algorithm, Dijkstra, Genetic Algorithm [42–44] and Ant Colony Optimization algorithm [45–47], are example of another algorithms have been used in multi agent centralized control. Oleiwi et al. [24] used a modified GA with A\* algorithm for its global motion controller while Atinc et al. [25] proposed a novel control scheme that has a centralized coverage control law.

The main issue in centralized control exists when the number of agents is expanding. The computation will become high since there is only one centralized processor that control over all of the system. Effect of this high computation, the time as well as the energy consumption will be effected at some point. Therefore, to solve this problem, hybrid control approach [5, 6, 24, 25] has been proposed with objective to balance between centralized control and distributed control [23, 26, 48–52]. Besides that, alternative towards optimizing or minimizing the trajectory length, time and energy consumption [24] as well as adding the intelligences [11, 20, 22, 27, 31, 32, 53] has taken into consideration to reduce the computation time. In terms of scalability, adaptability and flexibility of the controller can be claimed lesser as compared to distributed control. Any changes especially dealing with dynamics will cause the repetition in the computing and sometimes will effect overall of the system with only a limited number of controllers. Thus, centralized control sometimes does not fit with the dynamic environment.

Distributed control had proven scalable, adaptive, flexible and robust for multi agents system not only in static but also in a dynamic environment [54]. Many researchers had proven that their distributed controller can work efficiently for their multi agent robot systems [12, 26, 31, 32, 34, 48–50, 53, 55–58]. Innocenti et al. [27] have proven that their ActivMedia Pioneer 2DX mobile robots can reach its target by using their fuzzy logic controller. Same goes to Chen and Sun [21], Vatankhah et al. [22] and Glorennec [20] where they develop the distributed controller purposely for obstacles avoidance for their multi agents by using a fuzzy, neuro fuzzy, and a new optimal control protocol.

In distributed, the main issue is the task has to be distributed in a robust an efficient manner to ensure that every agent is able to perform its individual task cooperatively with another agents to achieve certain target. Distributing task among heterogeneous agents [11, 15] is more crucial and complex comparing with homogeneous agents which are identical [20–22, 59]. Limited sensing range and low bandwidth are also among physical constraints in distributed approach. With a limited local information, the agent cannot predict and cannot control the group behavior effectively in some sense. Another issues in distributed such as consensus, formation, containment, task allocation, optimization and intelligence will also discussed thoroughly in below section.

#### **3.2. Consensus**

**3. Problems and issues of cooperative multi-agent robot systems**

**Task Communication devices/network Interaction**

communication ranges

infinite series

Each agent communicates together (chains) using "hello-call" protocol to extend their effective

Using Wifi and three communication channels to interact between swarms for cooperation

Information spread by the effect of random walk and local communication known as information diffusion (equation of acquisition probability)

Communication by information probability by

Agent to agent

Agent to agent

One to many agents (broadcast)

Agent to groups of agents

rent approaches and to further expand the applications of MARS.

**3.1. Centralized and distributed control**

Cooperative multi-robot system using Hello-Call Communication [35]

8 Applications of Mobile Robots

networks [36]

Swarm robots control mobile robot using wireless sensor

Effect of grouping in local communication system of multiple mobile robots [37]

A design method of local communication area in multiple mobile robots systems [38, 39]

**Table 4.** Explicit communication researches.

Although researchers in recent years have addressed the issues of multi-agent robot systems (MARS), the current robot technology is still far from achieving many real world applications. Some real world MARS applications can be found in unmanned aerial vehicles (UAV's), unmanned ground vehicles (UGV's), unmanned underwater vehicles (UUV's), multi-robot surveillance, planetary exploration, search and rescue missions, service robots in smart homes and offices, warehouse management, as well as transportation. Therefore, in this paper, problems and issues related to cooperative multi-agent systems are discussed to improve the cur-

Based on Section 2.2, the differences between two types of control approaches have been highlighted. However, some problems and issues of both control system in coordinating multi agents will be discussed. By having centralized control, the global information of the environment has been used to calculate the path, trajectory or position of the agents before all [5, 6, 24–26, 40]. The information then can be sent directly to the agents by using a suitable communication medium. This is one advantage of this control where the agents can obtained the information directly from its central. Research by Azuma [33] shows that the central will sent the updated location directly to the agents by using a WIFI continuously until the agents reach the target point. The quadratic equation is used to calculate agent performances while Simultaneous Perturbation Stochastic Approximation is the algorithm used for the control design [41]. Besides that, A\* algorithm, Dijkstra, Genetic Algorithm [42–44] and Ant Colony Optimization algorithm [45–47], are example of another algorithms have been used in multi agent centralized control. Oleiwi et al. [24] used a modified GA with A\* algorithm for its global motion controller while Atinc et al. [25] proposed a novel control scheme that has a centralized coverage control law.

The main issue in centralized control exists when the number of agents is expanding. The computation will become high since there is only one centralized processor that control over all of Since multi-agent robots need to interact and communicate together to work cooperatively, issue on finding consensus for the homogeneous and heterogeneous robot has attracted researchers' attention over the past few years. Consensus refers to the degree of agreement among multi-agents to reach certain quantities of interest. The main problem of consensus control in multi-agent robots is to design a distributed protocol by using local information which can guarantee the agreements between robots to reach certain tasks or certain states. Therefore, a large number of interest concerning on developing the consensus control distributed protocol for homogeneous and heterogeneous robots which can be classified into a leader following consensus [60] and leaderless consensus [61–66], (to name a few), have been intensively studied by researchers recently [22, 67].

Each of heterogeneity agents is not identical and the states between agents are different which will cause difficulties in finding consensus. This is known as cooperative output consensus problem. This is a challenging issue for heterogeneous robots and there are a number of researchers who focused on the leaderless output consensus problem [13, 15, 68] and leaderfollower output consensus problem [13, 15, 69–71]. Wieland et al. [68] proposed an internal model principle to solve the leaderless output consensus problem for heterogeneous linear multi-agent systems. Wang et al. [69] discussed the classes of multi-agent system by switching topologies via static and dynamic feedback.

was also being explored by the authors [77]. For nonlinear systems, Liu et al. [50] investigated the distributed containment control problem for second order nonlinear multi-agents with dynamic leaders. The issues of containment problem for the first order and second order systems have been investigated by Ping and Wen [76], Bo et al. [51] and Rong et al. [52]. Ping and Wen [76] studied the first order multi-agent systems while Bo et al. [51] proposed the

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

http://dx.doi.org/10.5772/intechopen.79337

11

Formation control is an important issue to coordinate and control a group of multi-agent robots [49, 82–84]. The robots must be able to control their relative position and orientation among the robots in a group to move to a specific point. The motivations that drive the most attention among researchers to this problem are the biological inspirations, challenging control problems and the demand of multi-robot systems. There are many issues needed to be considered in designing a controller for mobile robot formation such as the stability of the formation, controllability of different formation patterns, safety and uncertainties in formations [82]. Other than that, issues of formation shape generation, formation reconfiguration and selection, formation tracking as well as role assignments in formation is discussed by Kiattisin.

There are three main control strategies for formation control proposed by previous researchers [82, 85, 86] such as (i) behavior based [87], (ii) virtual structure [88], (iii) leader-follower [89]. Each formation control method has its advantages and disadvantages. Balch and Arkin [87] proposed behavior-based formation control for their multi-robot teams. Behavior-based approach refers to several desired behavior of the agents such as goal seeking, obstacles avoidance, collision avoidance, etc. The final robot decision to choose which behavior comes first is based on the average weight of the behavior. The advantage of this approach is it can be used to guide the multi-agent robots in the unknown or dynamic environment by using the local information that the robot has. However, the drawback is where it cannot guarantee

Virtual structure is a formation control that considers the entire formation as a rigid body which was pioneered by Lewis and Tan [88]. The main advantage of this approach is easy coordination of the group's behavior and the formation is well maintain during maneuvers. However, the limitation of the virtual structures is it has to maintain the same virtual structure at all times especially when the formation shape needs to be frequently reconfigured. If not, the possible applications are limited. Leader and followers approach is another formation control for multiagent robots proposed by previous researchers [85, 86, 89]. In this strategy, some robots are considered as leaders while others will act as followers. The leaders will lead the followers to the target path while the followers will position and orientate by themselves while following the leaders. The main advantage of this approach is it can reduce the tracking error while the disadvantages are it will lead to a poor disturbance rejection property and the leader's motion will not depend on the followers. In addition, the formation does not tolerate to the leader's faults.

The networking system in formation control is another challenging issue highlighted by Chen and Wang [82] and Kiattisin in their reviewed papers. The communication delay in inter-robot information flow and communication loss problem will affect the performance of formation

control protocol for first order discrete-time systems with fixed time delays.

**3.4. Formation**

to converge easily during the process.

Research on finding consensus in the broadcasting area has also been carried out by few researchers. Li and Yan [72] solved the consensus in both fixing and switching type topology based on the spectrum radius of stochastic matrices. Azuma et al. [73] studied the consensus problem with a limited communication range and unlimited broadcast range by proposing its own controller. They introduced a concept of connected agent groups. This is to reduce consensus for "group to group" relation and for "agent to agent" relation in the groups by proposing two groups of consensus controller which are local and global. They proved that their controller can work efficiently in a mixed environment with communication and broadcast.

Besides that, research carried out by Das and Ghose [74, 75] solved the positional consensus problem for multi-agents. Das and Ghose [74] proposed a novel linear programming formulation and random perturbation input in the control command to achieve consensus at the prespecified location. The results showed that novel linear programming that is less intensive computation and perfect consensus can be obtained from random perturbation. They also proposed a novel linear programming formulation for their research [75]. Overall, it can be summarized that consensus problem is a vital issue which has been solved by many researchers. They have identified solutions to consensus problems for either homogeneous agents or heterogeneous agents which focus on finding an agreement among agents based on agent states (linear, nonlinear, static or dynamics topology) although there is an existence of leader in the environment or leaderless. Other than that, finding consensus in broadcasting topology/communication, broadcast mixed environment [73] and positioning agents [74] are also another recent issues focused by previous researchers.

#### **3.3. Containment**

Containment control is another problem investigated by many researchers. Containment problem refers to introducing more than one leader among the agents to ensure the groups are not ventured by the hazardous environment. If the agents are faced with this situation, they will move the robots to the safe region spanned by a group of leaders. The agents can either be homogeneous agents that have identical dynamics or heterogeneous agents that have different dynamics.

There are several issues investigated by previous researchers to solve the containment control problem for multi-agent robots such as (i) containment problem for a different dynamic level of the leaders and followers [70, 71, 14], (ii) containment problem for a linear and nonlinear systems [50, 76–79], (iii) containment problem for first order and second order systems [43–44, 72]. By assuming the follower and the leader of heterogeneous agents have different dynamics but the dynamics between each follower are similar, Youcheng and Yiguang [80] and Yuanshi and Long [81] had carried out their research studies. Besides that, Haghshenas et al. [14] solved the containment problem for two followers when the dynamic level is not identical.

A research on solving the containment problem for linear and nonlinear systems has been carried out by few researchers. Ping and Wen [76] investigated the linear first order systems for their multiple leaders. The distributed finite time containment problem for linear systems was also being explored by the authors [77]. For nonlinear systems, Liu et al. [50] investigated the distributed containment control problem for second order nonlinear multi-agents with dynamic leaders. The issues of containment problem for the first order and second order systems have been investigated by Ping and Wen [76], Bo et al. [51] and Rong et al. [52]. Ping and Wen [76] studied the first order multi-agent systems while Bo et al. [51] proposed the control protocol for first order discrete-time systems with fixed time delays.

#### **3.4. Formation**

model principle to solve the leaderless output consensus problem for heterogeneous linear multi-agent systems. Wang et al. [69] discussed the classes of multi-agent system by switching

Research on finding consensus in the broadcasting area has also been carried out by few researchers. Li and Yan [72] solved the consensus in both fixing and switching type topology based on the spectrum radius of stochastic matrices. Azuma et al. [73] studied the consensus problem with a limited communication range and unlimited broadcast range by proposing its own controller. They introduced a concept of connected agent groups. This is to reduce consensus for "group to group" relation and for "agent to agent" relation in the groups by proposing two groups of consensus controller which are local and global. They proved that their controller can work efficiently in a mixed environment with communication and broadcast. Besides that, research carried out by Das and Ghose [74, 75] solved the positional consensus problem for multi-agents. Das and Ghose [74] proposed a novel linear programming formulation and random perturbation input in the control command to achieve consensus at the prespecified location. The results showed that novel linear programming that is less intensive computation and perfect consensus can be obtained from random perturbation. They also proposed a novel linear programming formulation for their research [75]. Overall, it can be summarized that consensus problem is a vital issue which has been solved by many researchers. They have identified solutions to consensus problems for either homogeneous agents or heterogeneous agents which focus on finding an agreement among agents based on agent states (linear, nonlinear, static or dynamics topology) although there is an existence of leader in the environment or leaderless. Other than that, finding consensus in broadcasting topology/communication, broadcast mixed environment [73] and positioning agents [74] are also

Containment control is another problem investigated by many researchers. Containment problem refers to introducing more than one leader among the agents to ensure the groups are not ventured by the hazardous environment. If the agents are faced with this situation, they will move the robots to the safe region spanned by a group of leaders. The agents can either be homogeneous agents that have identical dynamics or heterogeneous agents that have different dynamics.

There are several issues investigated by previous researchers to solve the containment control problem for multi-agent robots such as (i) containment problem for a different dynamic level of the leaders and followers [70, 71, 14], (ii) containment problem for a linear and nonlinear systems [50, 76–79], (iii) containment problem for first order and second order systems [43–44, 72]. By assuming the follower and the leader of heterogeneous agents have different dynamics but the dynamics between each follower are similar, Youcheng and Yiguang [80] and Yuanshi and Long [81] had carried out their research studies. Besides that, Haghshenas et al. [14] solved the contain-

A research on solving the containment problem for linear and nonlinear systems has been carried out by few researchers. Ping and Wen [76] investigated the linear first order systems for their multiple leaders. The distributed finite time containment problem for linear systems

ment problem for two followers when the dynamic level is not identical.

topologies via static and dynamic feedback.

10 Applications of Mobile Robots

another recent issues focused by previous researchers.

**3.3. Containment**

Formation control is an important issue to coordinate and control a group of multi-agent robots [49, 82–84]. The robots must be able to control their relative position and orientation among the robots in a group to move to a specific point. The motivations that drive the most attention among researchers to this problem are the biological inspirations, challenging control problems and the demand of multi-robot systems. There are many issues needed to be considered in designing a controller for mobile robot formation such as the stability of the formation, controllability of different formation patterns, safety and uncertainties in formations [82]. Other than that, issues of formation shape generation, formation reconfiguration and selection, formation tracking as well as role assignments in formation is discussed by Kiattisin.

There are three main control strategies for formation control proposed by previous researchers [82, 85, 86] such as (i) behavior based [87], (ii) virtual structure [88], (iii) leader-follower [89]. Each formation control method has its advantages and disadvantages. Balch and Arkin [87] proposed behavior-based formation control for their multi-robot teams. Behavior-based approach refers to several desired behavior of the agents such as goal seeking, obstacles avoidance, collision avoidance, etc. The final robot decision to choose which behavior comes first is based on the average weight of the behavior. The advantage of this approach is it can be used to guide the multi-agent robots in the unknown or dynamic environment by using the local information that the robot has. However, the drawback is where it cannot guarantee to converge easily during the process.

Virtual structure is a formation control that considers the entire formation as a rigid body which was pioneered by Lewis and Tan [88]. The main advantage of this approach is easy coordination of the group's behavior and the formation is well maintain during maneuvers. However, the limitation of the virtual structures is it has to maintain the same virtual structure at all times especially when the formation shape needs to be frequently reconfigured. If not, the possible applications are limited. Leader and followers approach is another formation control for multiagent robots proposed by previous researchers [85, 86, 89]. In this strategy, some robots are considered as leaders while others will act as followers. The leaders will lead the followers to the target path while the followers will position and orientate by themselves while following the leaders. The main advantage of this approach is it can reduce the tracking error while the disadvantages are it will lead to a poor disturbance rejection property and the leader's motion will not depend on the followers. In addition, the formation does not tolerate to the leader's faults.

The networking system in formation control is another challenging issue highlighted by Chen and Wang [82] and Kiattisin in their reviewed papers. The communication delay in inter-robot information flow and communication loss problem will affect the performance of formation control and can even make the formation control system unstable. Therefore, a suitable communication protocol and network control system need to be implemented correctly into the robot system. In order to get more realistic formation control design for multi-agent robots coordination, the formation control needs to come together with an effective communication system design (either for local or global information via sensing or wireless network). Lastly, an alternative of implementing a hybrid control framework for multi-agent robots formation control has also become an issue to let the robots work in real world applications.

controllers and provide a sufficient condition for the controller gain to stabilize the broadcast for their group of Markov Chains [99]. Seyboth et al. [61] proposed the novel control strategy known as event-based broadcast control. They proved that their controller is more effective as compared to the time-based broadcast control. Finally, by having the intelligence, multi agent robot control is ready to be apply for an advance and complex multi-agents applications [3, 36, 49, 59]. As an example, Jolly et al. [53] and Candea et al. [3] have proposed their own

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

http://dx.doi.org/10.5772/intechopen.79337

13

Optimization is one of the important issue in designing a control system for multi agent robots. The objective is to find an optimal strategy under a given cost function either to find optimum trajectory/path, time, speed an as well as energy consumption. For example, by minimizing the path, less time is taken by the agent to move to its target point and the energy

Kumar and Kothare [100] have discovered the optimal strategy and optimal control architectures for their swarm agents. Their aim was to stabilize a swarm of stochastic agents by proposing the novel broadcast stochastic receding horizon controller. In order to search for an optimal path trajectory by minimizing the trajectory, time and energy consumption, Oleiwi et al. [24] had proposed the optimal motion planner. They combined the modified genetic algorithm with A\* algorithm to find a path from the start point to the goal point, fuzzy to

However, Chen and Sun [21] had a different approach, where they had proposed a new optimal control protocol to find an optimal control for their multi-agent consensus. Nagarajan and Thondiyath [92] proposed an algorithm that can minimize the turnaround time and cost during the agent's allocation task. The result showed that the algorithm performed better than

Issues on communications either implicit or explicit type of communication has been tackled since it will give effect to the multi agent controller performances. There are researchers who have focused on implicit communication where their robots interact based on sensor signal embedded to the robots [4, 30, 31]. However, there are also some drawbacks of implicit communication such as (1) limitations of the hardware and the sensors i.e. the hardware cannot support too many sensors, and the sensors can only work at certain conditions and distances, and (2) time delay if too many agents need to pass the information from one to another. Therefore, explicit communication come in to place where the information (messages) can be

However, other challenging parts of explicit communication are (1) to design a control framework to send the messages efficiently [33], (2) to design a suitable protocol that can guarantee all agents communicate effectively in the environment [3, 5, 34, 35], (3) to solve consensus problem which occurs during the interaction process either for homogeneous or heterogeneous agents [3], and (4) to design optimal controller that can optimize the speed and energy of the robots [33]. With the aim of providing an effective communication system for the robot

sent via broadcast (one to all) [5, 6, 33] or one to one agent [3, 5, 34].

avoid obstacles and cubic spline interpolation curve to reduce energy consumption.

controller to let the soccer robots coordinate and play successfully.

**3.7. Optimization**

the existing algorithm.

**3.8. Communications**

consumption will become less also.

#### **3.5. Task allocation**

The problem of task allocation among multi-agent robots has attracted researcher's attention. Once the computer assigns the task, the task needs to be sent to the robots for execution. Thus, a suitable approach needs to be applied in the system to ensure that the task is successfully allocated to the robots. Sarker et al. [90] used attractive field model to self-organize their robots while allocating its task. On the other side, Tolmidis and Petrou [91] proposed multi-objective optimization for their dynamic task allocation. The experiment results show a scalability, a generic solution and a better utilization of time as well as energy. Nagarajan and Thondiyath [92] also provided their own algorithm for task allocation which had proven better performances and better in minimizing the turnaround time, makespan and also cost.

#### **3.6. Intelligences**

The intelligence of multiagent robots to work cooperatively or coordinate its task is based on its controller. The design of the controller will determine agent's performances. The evolution of MARS shows that the level of intelligence is increasing in proportional with the technology. Since the beginning of artificial intelligences has been introduced, many researchers have started to design their controller by using this artificial intelligences approaches.

There are several approaches of artificial intelligences have been used by researchers in their multi agent controller development [11, 20, 22, 27, 31, 32, 53]. Fuzzy logic and neural network are approach have been used in multi agent robot control design which already proven its robustness and effectiveness [93]. Al-Jarrah et al. [31] used 2 fuzzy levels, which consisted of a fuzzy probabilistic control and adaptive neuro-fuzzy inference system, ANFIS. Vatankhah et al. [22] proposed a neuro-fuzzy structure with critic based learning structure and [11] proposed iterative learning control (ILC) scheme for their control system. Another researchers [20, 27, 32, 53, 94–97] were also using fuzzy control as one of the artificial intelligence approaches to develop their robots controller.

Other than artificial intelligence, there is another kind of intelligence proposed by previous researchers that had proven their multi-agent robots work effectively and successfully. Instead of focusing to a basic learning method, Tosic and Vilalta [98] proposed a unified framework for their multi-agent coordination by adopting the reinforcement learning, co-learning, and meta-learning in their system. Leader and follower concept also has been applied by few researchers to coordinate and plan their agent path [11, 22, 31]. Broadcast concept and framework for multi agent coordination can also be considered as an alternative towards intelligence [33, 61, 73, 99]. Azuma et al. [33, 73] developed the controller and broadcasted the signal from "agent to agent" or "agent to all agents". They also proposed integral-type broadcast controllers and provide a sufficient condition for the controller gain to stabilize the broadcast for their group of Markov Chains [99]. Seyboth et al. [61] proposed the novel control strategy known as event-based broadcast control. They proved that their controller is more effective as compared to the time-based broadcast control. Finally, by having the intelligence, multi agent robot control is ready to be apply for an advance and complex multi-agents applications [3, 36, 49, 59]. As an example, Jolly et al. [53] and Candea et al. [3] have proposed their own controller to let the soccer robots coordinate and play successfully.

#### **3.7. Optimization**

control and can even make the formation control system unstable. Therefore, a suitable communication protocol and network control system need to be implemented correctly into the robot system. In order to get more realistic formation control design for multi-agent robots coordination, the formation control needs to come together with an effective communication system design (either for local or global information via sensing or wireless network). Lastly, an alternative of implementing a hybrid control framework for multi-agent robots formation

The problem of task allocation among multi-agent robots has attracted researcher's attention. Once the computer assigns the task, the task needs to be sent to the robots for execution. Thus, a suitable approach needs to be applied in the system to ensure that the task is successfully allocated to the robots. Sarker et al. [90] used attractive field model to self-organize their robots while allocating its task. On the other side, Tolmidis and Petrou [91] proposed multi-objective optimization for their dynamic task allocation. The experiment results show a scalability, a generic solution and a better utilization of time as well as energy. Nagarajan and Thondiyath [92] also provided their own algorithm for task allocation which had proven better performances and better in minimizing the turnaround time, makespan and also cost.

The intelligence of multiagent robots to work cooperatively or coordinate its task is based on its controller. The design of the controller will determine agent's performances. The evolution of MARS shows that the level of intelligence is increasing in proportional with the technology. Since the beginning of artificial intelligences has been introduced, many researchers have

There are several approaches of artificial intelligences have been used by researchers in their multi agent controller development [11, 20, 22, 27, 31, 32, 53]. Fuzzy logic and neural network are approach have been used in multi agent robot control design which already proven its robustness and effectiveness [93]. Al-Jarrah et al. [31] used 2 fuzzy levels, which consisted of a fuzzy probabilistic control and adaptive neuro-fuzzy inference system, ANFIS. Vatankhah et al. [22] proposed a neuro-fuzzy structure with critic based learning structure and [11] proposed iterative learning control (ILC) scheme for their control system. Another researchers [20, 27, 32, 53, 94–97] were also using fuzzy control as one of the artificial intelligence

Other than artificial intelligence, there is another kind of intelligence proposed by previous researchers that had proven their multi-agent robots work effectively and successfully. Instead of focusing to a basic learning method, Tosic and Vilalta [98] proposed a unified framework for their multi-agent coordination by adopting the reinforcement learning, co-learning, and meta-learning in their system. Leader and follower concept also has been applied by few researchers to coordinate and plan their agent path [11, 22, 31]. Broadcast concept and framework for multi agent coordination can also be considered as an alternative towards intelligence [33, 61, 73, 99]. Azuma et al. [33, 73] developed the controller and broadcasted the signal from "agent to agent" or "agent to all agents". They also proposed integral-type broadcast

started to design their controller by using this artificial intelligences approaches.

approaches to develop their robots controller.

control has also become an issue to let the robots work in real world applications.

**3.5. Task allocation**

12 Applications of Mobile Robots

**3.6. Intelligences**

Optimization is one of the important issue in designing a control system for multi agent robots. The objective is to find an optimal strategy under a given cost function either to find optimum trajectory/path, time, speed an as well as energy consumption. For example, by minimizing the path, less time is taken by the agent to move to its target point and the energy consumption will become less also.

Kumar and Kothare [100] have discovered the optimal strategy and optimal control architectures for their swarm agents. Their aim was to stabilize a swarm of stochastic agents by proposing the novel broadcast stochastic receding horizon controller. In order to search for an optimal path trajectory by minimizing the trajectory, time and energy consumption, Oleiwi et al. [24] had proposed the optimal motion planner. They combined the modified genetic algorithm with A\* algorithm to find a path from the start point to the goal point, fuzzy to avoid obstacles and cubic spline interpolation curve to reduce energy consumption.

However, Chen and Sun [21] had a different approach, where they had proposed a new optimal control protocol to find an optimal control for their multi-agent consensus. Nagarajan and Thondiyath [92] proposed an algorithm that can minimize the turnaround time and cost during the agent's allocation task. The result showed that the algorithm performed better than the existing algorithm.

#### **3.8. Communications**

Issues on communications either implicit or explicit type of communication has been tackled since it will give effect to the multi agent controller performances. There are researchers who have focused on implicit communication where their robots interact based on sensor signal embedded to the robots [4, 30, 31]. However, there are also some drawbacks of implicit communication such as (1) limitations of the hardware and the sensors i.e. the hardware cannot support too many sensors, and the sensors can only work at certain conditions and distances, and (2) time delay if too many agents need to pass the information from one to another. Therefore, explicit communication come in to place where the information (messages) can be sent via broadcast (one to all) [5, 6, 33] or one to one agent [3, 5, 34].

However, other challenging parts of explicit communication are (1) to design a control framework to send the messages efficiently [33], (2) to design a suitable protocol that can guarantee all agents communicate effectively in the environment [3, 5, 34, 35], (3) to solve consensus problem which occurs during the interaction process either for homogeneous or heterogeneous agents [3], and (4) to design optimal controller that can optimize the speed and energy of the robots [33]. With the aim of providing an effective communication system for the robot coordination, researchers have tried to fix these problems by designing a suitable communication control that is relevant to the systems. There are also researchers who complement both communications implicitly and explicitly for their cooperative multi-agents research.

**4.2. Recommendations**

**Acknowledgements**

**Conflicts of interest**

been cited carefully.

**Author details**

Lumpur, Malaysia

**References**

research is gratefully acknowledged.

Zool Hilmi Ismail and Nohaidda Sariff\*

4-6 September 1989. 1989. pp. 283-290

\*Address all correspondence to: nohaiddasariff@yahoo.com

Some recommendations for cooperative MARS are as follows:

**1.** The reactive and deliberative control architectures have their own strengths and weaknesses. In the future, an effective way is to implement hybrid approach into MARS which consists of both reactive and deliberative control that leads to a more efficient system. **2.** An effective interaction between multi-agent robots can be achieved by integrating the implicit and explicit communications especially when the number of agents is increasing. **3.** A suitable communication protocol and network control system should be implemented into MARS to avoid time delay during transmission of information among agents.

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

http://dx.doi.org/10.5772/intechopen.79337

15

The financial support by Malaysian Government and University Teknologi Malaysia for this

Some works in this paper are based on study review from selected journals and proceedings regarding the cooperative multi-agent robot systems. All works from other researchers have

Centre for Artificial Intelligence & Robotics (CAIRO), Malaysia-Japan International Institute of Technology (MJIT), Universiti Teknologi Malaysia, Jalan Sultan Yahya Petra, Kuala

[1] Parker LE. ALLIANCE: An architecture for fault tolerant, cooperative control of heterogeneous mobile robots. IEEE Transactions on Robotics and Automation. 1998;**14**:220-240

[2] Asama H, Matsumoto A, Ishida Y. Design of an autonomous and distributed robot system: ACTRESS. In: IEEE/RSJ International Workshop on Intelligent Robots and Systems;

#### **4. Conclusion**

This paper has provided a review of cooperative multi-agent robots system (MARS). It shows that this research is leading to the creation of a robust cooperation and coordination of multiagent robots in various real applications. In order to produce high performance among agents, improvement in controller and communication part is the most crucial issues highlighted by researchers. Thus we strongly believe that this research has a potential to be expanded as the technology develops and the cooperative agents are foreseen to produce a big contribution towards the applications. Improvement on the controller design and communications either by adding intelligences or optimize certain cost function is in parallel with the technologies development which will then produce a multi agents which are mobile, scalable, flexible, global, dynamic and persistent connectivity. Regardingly, the following are other future challenges and recommendations that could be explored by our future researchers in expanding the area of MARS.

#### **4.1. Future challenges**

There are many challenges for future cooperative multi-agent systems and, among them, the most crucial challenge lies in controller design, which should be robust and intelligent enough to support overall system. Besides that, communication among agents is also important since it will determine the success of the system. Therefore, there are several future challenges that should be taken into consideration:


#### **4.2. Recommendations**

coordination, researchers have tried to fix these problems by designing a suitable communication control that is relevant to the systems. There are also researchers who complement both

This paper has provided a review of cooperative multi-agent robots system (MARS). It shows that this research is leading to the creation of a robust cooperation and coordination of multiagent robots in various real applications. In order to produce high performance among agents, improvement in controller and communication part is the most crucial issues highlighted by researchers. Thus we strongly believe that this research has a potential to be expanded as the technology develops and the cooperative agents are foreseen to produce a big contribution towards the applications. Improvement on the controller design and communications either by adding intelligences or optimize certain cost function is in parallel with the technologies development which will then produce a multi agents which are mobile, scalable, flexible, global, dynamic and persistent connectivity. Regardingly, the following are other future challenges and recommendations that could be explored by our future researchers in expanding the area of MARS.

There are many challenges for future cooperative multi-agent systems and, among them, the most crucial challenge lies in controller design, which should be robust and intelligent enough to support overall system. Besides that, communication among agents is also important since it will determine the success of the system. Therefore, there are several future challenges that

**1.** The need of more powerful coordination among homogeneous and heterogeneous agents. This is especially for advance and complex multi-agent robots application such as soccer

**2.** Since the physical identity and capability among heterogeneous agents are not identical, issues in coordinating the agents will become more challenging compared to homogene-

**3.** Adapting various artificial intelligence approaches in solving control and communication problems of MARS either consensus [13, 15, 69–71], containment [70, 71, 14, 49, 82–84], position [45, 58] or any other problems should be considered as long as there is an improve-

**4.** Issues in reducing the energy consumption and time travel will produce an optimal controller for the agents. Thus, an appropriate design of controller should be applied together

**5.** By broadcasting the information to agents, the information can be sent directly to agents, to avoid losses and time delay during transmission of the information [33, 45, 47–48, 51, 52]. Thus, this research should be expanded since the communication among agents

robots [3], swarm robots [36], UGV's [4], UAV's [4] or any other robots.

ous agents [2–5, 6, 9, 12]. Attention should be given more to these agents.

with the suitable communication system that can support the MARS [3, 5].

communications implicitly and explicitly for their cooperative multi-agents research.

**4. Conclusion**

14 Applications of Mobile Robots

**4.1. Future challenges**

should be taken into consideration:

ment towards the robot performances.

can be improved from time to time.

Some recommendations for cooperative MARS are as follows:


#### **Acknowledgements**

The financial support by Malaysian Government and University Teknologi Malaysia for this research is gratefully acknowledged.

#### **Conflicts of interest**

Some works in this paper are based on study review from selected journals and proceedings regarding the cooperative multi-agent robot systems. All works from other researchers have been cited carefully.

#### **Author details**

Zool Hilmi Ismail and Nohaidda Sariff\*

\*Address all correspondence to: nohaiddasariff@yahoo.com

Centre for Artificial Intelligence & Robotics (CAIRO), Malaysia-Japan International Institute of Technology (MJIT), Universiti Teknologi Malaysia, Jalan Sultan Yahya Petra, Kuala Lumpur, Malaysia

#### **References**


[3] Candea C, Hu H, Iocchi L, Nardi D, Piaggio M. Coordination in multi agent RoboCup teams. Robotics and Autonomous Systems. 2001;**36**:67-86

[20] Glorennec PY. Coordination between autonomous robots. International Journal of

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

http://dx.doi.org/10.5772/intechopen.79337

17

[21] Chen Y, Sun J. Distributed optimal control for multi agent systems with obstacles avoid-

[22] Vatankhah R, Etemadi S, Alasty A, Vossoughi G. Adaptive critic based neuro fuzzy controller in multi agents: Distributed behavioral control and path tracking. Neuro-

[23] Franco ML, Sanchez EN, Alanis AY, Franco CL, Daniel NA. Decentralized control for stabilization of nonlinear multi agent systems using neural inverse optimal control.

[24] Oleiwi BK, Al-Jarrah R, Roth H, Kazem BI. Integrated motion planing and control for multi objectives optimization and multi robots navigation. In: 2nd IFAC Conference on Embedded Systems, Computer Intelligence and Telematics CESCIT 2015. Vol. 48. 2015.

[25] Atinc GM, Stipanovic DM, Voulgaris PG. Supervised coverage control of multi agent

[26] Posadas JL, Poza JL, Simo JE, Benet G, Blanes F. Agent based distributed architecture for mobile robot control. Engineering Applications of Artificial Intelligence. 2008;**21**:805-823

[27] Innocenti B, Lopez B, Salvi J. A multi agent architecture with cooperative fuzzy control

[28] Kudelski M, Gambardella LM, Caro GAD. RoboNetSim: An integradted framework for multi robot and network simulation. Robotics and Autonomous Systems. 2013;**61**:483-496

[29] Couceiro MS, Vargas PA, Rocha RP. Bridging the reality gap between the webots simulator and E-puck robots. Robotics and Autonomous Systems. 2014;**62**:1549-1567

[30] Kuniyoshi Y, Riekki J, Rougeaux MIS. Vision based behaviours for multi robot cooperation. In: Proceedings of the IEEE/RSJ/GI International Conference on Intelligent Robots and Systems '94 'Advanced Robotic Systems and the Real World' IROS '94. Vol. 2. 1994.

[31] Al-Jarrah R, Shahzad A, Roth H. Path planning and motion coordination fro multi robot system using probabilistic neuro fuzzy. In: 2nd IFAC Conference on Embedded Systems, Computer Intelligence and Telematics CESCIT 2015. Vol. 48; 22-24 June 2015. pp. 46-51

[32] Pham DT, Awadalla MH, Eldukhri EE. Fuzzy and neuro fuzzy based cooperative mobile robots. In: 2nd I\*PROMS Virtual International Conference; 3-14 July 2006. pp. 578-583

[33] Azuma S, Yoshimura R, Sugie T. Broadcast control of multi agents systems. Automatica.

[34] Wang J. On sign board based inter robot communication in distributed robotics systems. In: IEEE International Conference on Robotics and Automation. 1994. pp. 1045-1050

for a mobile robot. Robotics and Autonomous Systems. 2007;**55**:881-891

ance. Neurocomputing. 15 January 2016;**173**(Part 3):2014-2021

Approximate Reasoning. 1997;**17**:433-446

computing. 2012;**88**:24-35

pp. 99-104

pp. 923-931

2013;**49**:2307-2316

Neurocomputing. 2015;**168**:81-91

systems. Automatica. 2014;**50**:2936-2942


[20] Glorennec PY. Coordination between autonomous robots. International Journal of Approximate Reasoning. 1997;**17**:433-446

[3] Candea C, Hu H, Iocchi L, Nardi D, Piaggio M. Coordination in multi agent RoboCup

[4] Rosa L, Cognetti M, Nicastro A, Alvarez P, Oriolo G. Multi task cooperative control in a heterogeneous ground air robot team. In: 3rd IFAC Workshop on Multivehicle Systems.

[5] Brunete A, Hernando M, Gambao E, Torres JE. A behavior based control architecture for heterogeneous modular, multi configurable, chained micro robots. Robotics and

[6] Simonin O, Grunder O. A cooperative multi robot architecture for moving a paralyzed

[7] Cao YU, Fukunagu AS, Kahng AB. Cooperative mobile robotics: Antecedents and direc-

[8] Yan Z, Jouandeau N, Cherif AA. A survey and analysis of multi robot coordination.

[9] Sugawara K, Sano M. Cooperative acceleration of task performances: Foraging behavior

[10] Hackwood S, Beni G. Self organization of sensors for swarm intelligence. In: IEEE Inter-

[11] Li J. Iterative learning control approach for a kind of Heterogeneous multi agent systems with distributed initial state learning. Applied Mathematics and Computation. 2015;

[12] Lope JD, Maravall D, Quinonez Y. Self-organizing techniques to improve the decentralized multi task distribution in multi robot systems. Neurocomputing. 2015;**163**:47-55 [13] Ma Q, Miao G. Output consensus for heterogeneous multi agent systems with linear

[14] Haghshenas H, Badamchizadeh MA, Baradarannia M. Containment control of hetero-

[15] Li Z, Duan Z, Lewis FL. Distributed robust consensus control of multi agent systems

[16] Vlacic L, Engwirda A, Kajitani M. Cooperative behavior of intelligent agents: Theory and practice. In: Sinha NK, Gupta MM, Zadeh LA, editors. Soft Computing & Intelligent

[17] Oliveira E, Fischer K, Stepankova O. Multi Agent Systems: Which Research for Which

[18] Parker LE. Heterogeneous Multi Robot Cooperation. MA, USA: Massachusetts Institute

[19] Goldberg D. Heterogeneous and homogeneous robot group behavior. In: Proceedings

teams. Robotics and Autonomous Systems. 2001;**36**:67-86

Vol. 48. 2015. pp. 53-58

16 Applications of Mobile Robots

**265**:1044-1057

Autonomous Systems. 2012;**60**:1607-1624

tions. Journal of Autonomous Robots. 1997;**4**:1-23

International Journal of Advanced Robotics Systems. 2013;**10**:1-18

of interacting multi robots system. Physica D. 1997;**100**:343-354

national Conference on Robotics and Automation. 1992. pp. 819-829

dynamics. Applied Mathematics and Computation. 2015;**271**:548-555

geneneous linear multi agent systems. Automatica. 2015;**54**:210-216

Applications. Robotics and Autonomous Systems. 1999;**27**:91-106

Systems. UK: Academic Press; 2000. pp. 279-307

of Techology Cambridge; 1994

AAAI-96. 1996. p. 1390

with heterogeneous matching uncertainties. Automatica. 2014;**50**:883-889

robot. Mechatronics. 2009;**19**:463-470


[35] Ichikawa S, Hara F, Hosokai H. Cooperative route searching behavior of multi robot system using hello call communication. In: Proceedings of the 1993 IEEE/RSJ International Conference on Intelligent Robots and Systems; 26-30 July 1993. pp. 1149-1156

[47] Sariff N, Buniyamin N. Ant colony system for robot path planning in global static environment. In: Selected Topics in System Science & Simulation in Engineering. World

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

http://dx.doi.org/10.5772/intechopen.79337

19

[48] Liu T, Jiang ZP. Distributed nonlinear control of mobile autonomous multi agents.

[49] Peng Z, Yang S, Wen G, Rahmani A, Yu Y. Adaptive distributed formation control for multiple nonholonomic wheeled mobile robots. Neurocomputing. 15 January

[50] Liu Z, Jin Q, Chen Z. Distributed containment control for bounded unknown second order nonlinear multi agent systems with dynamic leaders. Neurocomputing. 2015;**168**:

[51] Bo L, Qiang CZ, Xin LZ, Yan ZC, Qing Z. Containment control of multi agent systems with fixed time delays in fixed directed networks. Neurocomputing. 15 January 2016;

[52] Rong L, Shen H, Lu J, Li J. Distributed reference model based containment control of

[53] Jolly KG, Kumar RS, Vijayakumar R. Intelligent task planning and action selection of a mobile robot in a multi agent system through a fuzzy neural network approach.

[54] Ren W, Cao Y. Overview of recent research in distributed muti-agent coordination. Distributed Coordination of Multi-agent Networks Emergent Problems, Models and

[55] Buniyamin N, Sariff N, Wan Ngah WAJ, Mohamad Z. A simple local path planning algorithm for autonomous mobile robots. International Journal of Systems Applications,

[56] Sariff N, Elyana N. Mobile robot obstacles avoidance by using braitenberg approach. In: 2nd International Conference on Emerging Trends in Scientific Research (ICETSR);

[57] Mohamed F, Sariff N, ZainalAbidin IZ. Low cost serving robot using fuzzy logic techniques. In: Proceedings of the Second International Conferences on Advances in Auto-

[58] Hakim Z, Sariff N, Buniyamin N. The development of a low cost remote control partner lawnmower robot. In: 4th Student Conference on Research and Development (SCORED

[59] Farina M, Perizzato A, Scattolini R. Application of distributed predictive control to motion and coordination problems for unicycle autonmous robots. Robotics and Auto-

mation and Robotics (AAR 2013); Kuala Lumpur, Malaysia; May 2013

second-order multi agent systems. Neurocomputing. 2015;**168**:254-259

Engineering Applications of Artificial Intelligence. 2010;**23**:923-933

Engineering & Development (ISAED 2011). 2011;**5**:151-159

Kuala Lumpur, Malaysia; November 2014

2006). June 2006. pp. 152-155

nomous Systems. 2015;**72**:248-260

Scientific and Engineering Academic and Society (WSEAS); 2010, pp 192-197

Automatica. 2014;**50**:1075-1086

2016;**173**(Part 3):1485-1494

**173**(Part 3):2069-2075

Issues; 2011:23-41

1138-1143


[47] Sariff N, Buniyamin N. Ant colony system for robot path planning in global static environment. In: Selected Topics in System Science & Simulation in Engineering. World Scientific and Engineering Academic and Society (WSEAS); 2010, pp 192-197

[35] Ichikawa S, Hara F, Hosokai H. Cooperative route searching behavior of multi robot system using hello call communication. In: Proceedings of the 1993 IEEE/RSJ International

[36] Li W, Shen W. Swarm behavior control of mobile multi-robots with wireless sensor net-

[37] Yoshida E, Arai T, Ota J, Miki T. Effect of grouping in local communication system of multiple mobile robots. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems '94 'Advanced Robotic Systems and the Real World' IROS '94. 1994.

[38] Yoshida E, Yamamoto M, Arai T, Ota J, Kurabayashi D. A design method of local communication area in multiple mobile robot system. In: IEEE International Conference on

[39] Yoshida E, Yamamoto M, Arai T, Ota J, Kurabayashi D. A design method of local communication range in multiple mobile robot system. In: IEEE International Conference on

[40] Sariff N, Buniyamin N. An overview of autonomous robot path planning algorithms. In: 4th Student Conference on Research and Development (SCORED 2006); Shah Alam,

[41] Sariff N, Ismail ZH. Investigation of simultaneous pertubation stochastic algorithm parameters effect towards multi agent robot motion coordination performances. In: 2017 IEEE 7th International Conference on Underwater System Technology: Theory and Applications, Universiti Teknologi Malaysia; Kuala Lumpur. December 2017. pp. 1-6 [42] Sariff N, Buniyamin N. Evaluation of robot path planning algorithms in global static environments: Genetic algorithm VS ant colony optimization algorithm. International

Journal of Electrical and Electronic Systems Research (IEESR 2010). 2010;**3**:1-12

[43] Sariff N, Buniyamin N. Genetic algorithm versus ant colony optimization algorithm: Comparison of performances in robot path planning application. In: 7th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2010);

[44] Sariff N, Buniyamin N. Comparative study of genetic algorithm and ant colony optimization algorithm in global static environment of different complexities. In: 2009 IEEE International Symposium on Computational Intelligence in Robotics and Automation

[45] Buniyamin N, Sariff N, Wan Ngah WAJ, Mohamad Z. Robot global path planning overview and a variation of ant colony system algorithm. International Journal of

[46] Sariff N, Buniyamin N. Ant colony system for robot path planning in global static environment. In: 9th International Conference on System Science and Simulation in

Mathematics and Computers in Simulation (IMACS 2011). 2011;**5**:9-16

Engineering (ICOSSSE'10); Iwate, Japan. October 2010. pp. 1-6

Conference on Intelligent Robots and Systems; 26-30 July 1993. pp. 1149-1156

works. Journal of Network and Computer Applications. 2011;**34**:1398-1407

Robotics and Automation. 1995. pp. 2567-2572

Robotics and Automation. 1995. pp. 274-279

Madeira, Portugal. June 2010. pp. 125-132

(CIRA 2009); Daejeon, Korea. December 2009. pp. 132-137

Malaysia. June 2006. pp. 184-188

pp. 808-815

18 Applications of Mobile Robots


[60] Peng K, Yang Y. Leader-following consensus problem with a varyning velocity leader and time-varying delays. Physica D: Statictical Mechanics and Its Applications. 2009;**388**: 193-208

[76] Ping HJ, Wen YH. Collective coordination of multi-agent systems guided by multiple

A Survey and Analysis of Cooperative Multi-Agent Robot Systems: Challenges and Directions

http://dx.doi.org/10.5772/intechopen.79337

21

[77] Wang X, Li S, Shi P. Distributed finite-time containment control for double integrator

[78] Mei J, Ren W, Ma BLG. Containment control for networked unknown langrangian systems with multiple dynamic leaders under a directed graph. In: Proceedings of the

[79] Liu H, Cheng L, Hao MTZ. Containment control of double-integrator multi-agent systems with a periodic sampling: A small-gain theorem based method. Proceedings of

[80] Youcheng L, Yiguang H. Multi-leader set coordination of multi agent systems with random switching topologies. In: Proceedings of the IEEE International Conference on

[81] Yuanshi Z, Long W. Containment control of heterogeneous multi-agent systems. Inter-

[82] Chen YQ, Wang Z. Formation control: A review and a new consideration. In: 2005 IEEE/ RSJ International Conference on Intelligent Robots and Systems. 2005. pp. 3664-3669 [83] Oh KK, Park MC, Ahn HS. A survey of multi agent formation control. Automatica.

[84] Nascimento TP, Moreira AP, Conceicao AGS. Multi robot nonlinear model predictive formation control: Moving target and target absence. Robotics and Autonomous

[85] Consolini L, Morbidi F, Prattichizzo D, Tosques M. A geometric characterization of leader follower formation control. In: IEEE International Conference on Robotics and

[86] Consolini L, Morbidi F, Prattichizzo D, Tosques M. Leader follower formation control as a disturbance decoupling problem. In: European Control Conference (ECC). 2007.

[87] Balch T, Arkin RC. Behaviour-based formation control for multi-robots teams. IEEE

[88] Lewis, Tan. High precision formation control of mobile robots using virtual structures.

[89] Das AK, Fierro R, Kumar V, Ostrowski JP, Spletzer J, Taylor CJ. A vision based formation control framework. IEEE Transactions on Robotics and Automation. 2002;**18**:813-825 [90] Sarker MOF, Dahl TS, Arcaute E, Christensen K. Local interactions over global broadcasts for improves task allocation in self organized multi robot systems. Robotics and

Transactions on Robotics and Automation. 1998;**14**:926-939

multi-agent systems. IEEE Transactions on Cybernetics. 2014;**44**:1518-1528

33nd Chinese Control Conference; Nanjing, China. 2014. pp. 1407-1412

leaders. Chinese Pyhsics B. 2009;**18**:3777-3782

American Control Conference. 2013. pp. 522-527

Decision and Control. 2010. pp. 3820-3825

national Journal of Control. 2014;**87**:1-8

2015;**53**:424-440

pp. 1492-1497

Systems. 2013;**61**:1502-1515

Automation. 2007. pp. 2397-2402

Autonomous Robots. 1997;**4**:387-403

Autonomous Systems. 2014;**62**:1453-1462


[76] Ping HJ, Wen YH. Collective coordination of multi-agent systems guided by multiple leaders. Chinese Pyhsics B. 2009;**18**:3777-3782

[60] Peng K, Yang Y. Leader-following consensus problem with a varyning velocity leader and time-varying delays. Physica D: Statictical Mechanics and Its Applications. 2009;**388**:

[61] Seyboth GS, Dimarogonas DV, Johansson KH. Event based broadcasting for multi agent

[62] Saber RO, Fax JA, Murray RM. Consensus and coooperation in networked multi agent

[63] Zhu W, Jiang ZP, Feng G. Event-based consensus of multi-agent systems with general

[64] Ren W, Beard RW, Atkins EM. Information consensus in multivechicle cooperative con-

[65] Wang A. Event based consensus control for single integrator networks with communica-

[66] Hou B, Sun F, Li H, Chen Y, Liu G. Observer based cluster consensus control of high

[67] Nowzari C, Cortes J. Zeno-free, distributed event triggered communication and control for multi agent average consensus. In: 2014 American Control Conference (ACC); June

[68] Wieland P, Sepulchre R, Allgower F. An internal model principle is necessary and suf-

[69] Wang X, Ji H, Wang C. Distributed output regulation of leader follower multi agents systems. International Journal of Robust and Nonlinear Control. 10 January 2013;**23**(1):

[70] Su Y, Huang J. Cooperative output regulation of linear multi-agent systems. IEEE

[71] Lee TH, Park JH, Ji DH, Jung HY. Leader following consensus problem of heterogeneous multi-agent systems with nonlinear dynamics using fuzzy disturbance observer.

[72] Li J, Yan W. Consensus problems for multi agent system with broadcasting type topology. In: 2012 Second International Conference on Instrumentation & Measurement,

[73] Azuma S, Yoshimura R, Sugie T. Multi-agent consensus under a communication broadcast mixed environment. International Journal of Control. 2014;**87**:1103-1116

[74] Das K, Ghose D. Positional consensus in multi agent systems using a broadcast control mechanism. In: 2009 American Control Conference; 10-12 June 2009. pp. 5731-5736 [75] Das K, Ghose D. Broadcast control mechanism for positional consensus in multiagent system. IEEE Transactions on Control Systems Technology. 5 Sept. 2015;**23**(5):1807-1826

tion time delay. Neurocomputing. 15 January 2016;**173**(part 3):1715-1719

ficient for linear output synchronization. Automatica. 2011;**47**:1068-1074

order multi agent systems. Neurocomputing. 2015;**168**:979-982

Transactions on Automatic Control. 2012;**57**:1062-1066

Computer, Communication and Control. 2012. pp. 967-970

average consensus. Automatica. 2013;**49**:245-252

systems. Proceedings of the IEEE. 2007;**95**:215-233

linear models. Automatica. 2014;**50**:552-558

trol. IEEE in Control Systems. 2007;**27**:71-82

193-208

20 Applications of Mobile Robots

2014. pp. 4-6

48-66

Complexity. 2014;**19**:20-31


[91] Tolmidis AT, Petrou L.Multi objective optimization for dynamic task allocation in a multi robot system. Engineering Applications of Artificial Intelligence. 2013;**26**:1458-1468

**Chapter 2**

Provisional chapter

**Motion Control and Velocity-Based Dynamic**

DOI: 10.5772/intechopen.79397

The design of motion controllers for wheeled mobile robots is often based only on the robot's kinematics. However, to reduce tracking error it is important to also consider the robot dynamics, especially when high-speed movements and/or heavy load transportation are required. Commercial mobile robots usually have internal controllers that accept velocity commands, but the control signals generated by most dynamic controllers in the literature are torques or voltages. In this chapter, we present a velocity-based dynamic model for differential-drive mobile robots that also includes the dynamics of the robot actuators. Such model can be used to design controllers that generate velocity commands, while compensating for the robot dynamics. We present an explanation on how to obtain the parameters of the dynamic model and show that motion controllers designed for the robot's kinematics can be easily integrated with the velocity-based dynamic compensation controller. We conclude the chapter with experimental results of a trajectory tracking controller that show a reduction of up to 50% in tracking error index IAE due to the

Keywords: velocity-based dynamic model, dynamic modeling, dynamic compensation,

A common configuration for mobile robots is the differential drive, which has two independently driven parallel wheels and one (or more) unpowered wheel to balance the structure [1]. For several years differential-drive mobile robots (DDMR) have been widely used in many applications because of their simple configuration and good mobility. Some applications of

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

Motion Control and Velocity-Based Dynamic

**Compensation for Mobile Robots**

Compensation for Mobile Robots

Additional information is available at the end of the chapter

application of the dynamic compensation controller.

motion control, tracking control

Additional information is available at the end of the chapter

Felipe Nascimento Martins and Alexandre Santos Brandão

Felipe Nascimento Martins and Alexandre Santos Brandão

http://dx.doi.org/10.5772/intechopen.79397

Abstract

1. Introduction


#### **Chapter 2** Provisional chapter

#### **Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots** Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

DOI: 10.5772/intechopen.79397

Felipe Nascimento Martins and Alexandre Santos Brandão Felipe Nascimento Martins and Alexandre Santos Brandão

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79397

#### Abstract

[91] Tolmidis AT, Petrou L.Multi objective optimization for dynamic task allocation in a multi robot system. Engineering Applications of Artificial Intelligence. 2013;**26**:1458-1468 [92] Nagarajan T, Thondiyath A. Heuristic based task allocation algorithm for multiple robots using agents. In: International Conference on Design and Manufacturing

[93] Sariff N, Nadihah NH. Automatic mobile robot obstacles avoidances in a static environment using hybrid approaches (fuzzy logic and artificial neural network). In: 2014 International Conference Artificial Intelligence System Technology (ICAIST); Kota

[94] Mohamed F, Sariff N, Abidin IZZ. Low cost serving robot using fuzzy logic techniques. International Journal of Advancements in Mechanical and Aeronautical Engineering

[95] Mohamad MF, Sariff N, Buniyamin N. Mobile robot obstacle avoidance in various type of static environments using fuzzy logic approach. In: 2014 International Conference on

[96] Akmal Jeffril M, Sariff N. The integration of fuzzy logic and artificial neural network method for mobile robot obstacles avoidance in a static environment. In: 2013 IEEE 3rd International Conferences on System Engineering and Technology (ICSET); Shah Alam,

[97] Hajar Ashikin S, Akmal Jeffril M, Sariff N. Mobile robot obstacles avoidances by using fuzzy logic techniques. In: 2013 IEEE 3rd International Conferences on System

[98] Tosic PT, Vilalta R. A unified farmework for reinforcement learning, co-learning and meta-learning how to coordinate in collaborative multi agents systems. In: International

[99] Azuma S, Yoshimura R, Sugie T. Broadcast control of group of Markov chains. In: 51st IEEE Conference on Decision and Control; 10-13 December 2012. pp. 2059-2064 [100] Kumar G, Kothare MV. Broadcast stochastic receding horizon control of multi agent

Engineering and Technology (ICSET); Shah Alam, Malaysia. 2013. pp. 332-335

Conference on Computational Science ICCS. Vol. 1. 2012. pp. 2217-2226

Electrical, Electronics and System Engineering (ICEESE2014); December 2014

IConDM. Vol. 64. 2013. pp. 844-853

Kinabalu, Sabah; December 2014

Malaysia. August 2013. pp. 326-330

systems. Automatica. 2013;**49**:3600-3606

(IJAMAE). 2013;**1**:54-58

22 Applications of Mobile Robots

The design of motion controllers for wheeled mobile robots is often based only on the robot's kinematics. However, to reduce tracking error it is important to also consider the robot dynamics, especially when high-speed movements and/or heavy load transportation are required. Commercial mobile robots usually have internal controllers that accept velocity commands, but the control signals generated by most dynamic controllers in the literature are torques or voltages. In this chapter, we present a velocity-based dynamic model for differential-drive mobile robots that also includes the dynamics of the robot actuators. Such model can be used to design controllers that generate velocity commands, while compensating for the robot dynamics. We present an explanation on how to obtain the parameters of the dynamic model and show that motion controllers designed for the robot's kinematics can be easily integrated with the velocity-based dynamic compensation controller. We conclude the chapter with experimental results of a trajectory tracking controller that show a reduction of up to 50% in tracking error index IAE due to the application of the dynamic compensation controller.

Keywords: velocity-based dynamic model, dynamic modeling, dynamic compensation, motion control, tracking control

#### 1. Introduction

A common configuration for mobile robots is the differential drive, which has two independently driven parallel wheels and one (or more) unpowered wheel to balance the structure [1]. For several years differential-drive mobile robots (DDMR) have been widely used in many applications because of their simple configuration and good mobility. Some applications of

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

DDMR are surveillance [2], floor cleaning [3], industrial load transportation [4], autonomous wheelchairs [5], and others.

<sup>τ</sup><sup>∈</sup> <sup>R</sup><sup>r</sup>�<sup>1</sup> is the vector of input torques, where <sup>r</sup> is the number of inputs, B qð Þ<sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>r</sup> is the input transformation matrix, <sup>λ</sup><sup>∈</sup> <sup>R</sup><sup>m</sup>�<sup>1</sup> is the vector that represents restriction forces, and A qð Þ<sup>∈</sup> <sup>R</sup><sup>m</sup>�<sup>n</sup> is the matrix associated to such restrictions. Two well known properties of such model are [9, 22]:

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

http://dx.doi.org/10.5772/intechopen.79397

25

The above properties are widely used on the development and stability analysis of controllers for mobile robots, as shown in [8, 9, 22, 23]. But, such controllers generate torque commands, not velocities, as usually accepted by commercial robots. The conversion from torque to velocity commands requires knowledge of the actuation system of the robot (model of its motors and its speed controllers). On the other hand, a controller designed from a velocitybased dynamic model generates linear and angular velocities that can be directly applied as

In such a context, now the dynamic model for the DDMR proposed in [16] is reviewed. For convenience, we first present its equations again. Then, the dynamic model is written in such a way that it becomes similar to the classical dynamic equation based on torques. Figure 1 depicts a DDMR with the variables of interest. There, u and ω are, respectively, the linear and angular velocities, G is the center of mass, h is the point of interest (whose position should be controlled)

Figure 1. The differential drive mobile robot (DDMR). u and ω are, respectively, the linear and angular velocities, G is the center of mass, h is the point of interest with coordinates x and y in the XY plane, ψ is the robot orientation, a is the distance from the point of interest to the point in the middle of the virtual axle that links the traction wheels (point B), b is the distance between points G and B, and d is the distance between the points of contact of the traction wheels to the floor.

1. M qð Þ is a symmetric and positive definite matrix, that is, M qð Þ¼ M qð Þ<sup>T</sup> <sup>&</sup>gt; 0;

2. <sup>M</sup>\_ � 2Vm

is antisymmetric.

commands for mobile robots.

In the literature, most of the motion controllers for DDMR are based only on its kinematics. The main reasons for that are: (a) the kinematic model is simpler than the dynamic model, therefore the resulting controllers are less complex and simpler to tune; (b) the accuracy of the dynamic model depends on several parameters that might change or are difficult to measure, like the robot's mass and moment of inertia; and (c) dynamic controllers usually generate torque or voltage commands, while mobile robots frequently have internal velocity controllers that take velocity as input [6]. However, the robot's low-level velocity control loops do not guarantee perfect velocity tracking, especially when high-speed movements and/or heavy load transportation are required. In such cases, to reduce tracking error, it becomes essential to consider the robot dynamics as well, as shown in [7].

A possible solution to overcome the problem described above is to design a controller that compensates for the robot's dynamics. Commercial mobile robots usually have internal controllers that accept velocity commands, like the Pioneer 3 from Adept Mobile Robots, the Khepera from K-Team Corporation, and the robuLAB-10 from Robosoft Inc. However, the control signals generated by most dynamic controllers in the literature are torques or voltages, as in [8–14]. Because of that, some researchers have proposed dynamic controllers that generate linear and angular velocities as commands [15, 16]. In some works, the dynamic model is divided in to two parts, allowing the design of independent controllers for the robot kinematics and dynamics [17–20]. Finally, to reduce performance degradation in applications in which the robot dynamic parameters may vary (such as load transportation) or when the knowledge of the dynamic parameters is imprecise, adaptive controllers can also be considered [7, 21].

The above-mentioned works applied a dynamic model that has linear and angular velocities as inputs, which illustrates the interest on such kind of dynamic model. In such context, this chapter explains the velocity-based dynamic model and its mathematical properties, which are useful for the design of controllers that compensate for the robot dynamics. It also illustrates how to design a trajectory tracking motion controller based on the robot's kinematics, and how to integrate it with a velocity-based dynamic compensation controller.

#### 2. Dynamic model

The classical equation to represent the dynamics of mobile robots can be obtained via Lagrangian formulation, resulting in [22].

$$\mathbf{M(q)}\ddot{\mathbf{q}} + \mathbf{V\_m(q, \dot{q})}\dot{\mathbf{q}} + \mathbf{F\_m(\dot{q})} + \mathbf{G\_m(q)} + \boldsymbol{\tau\_d} = \mathbf{B(q)}\boldsymbol{\tau} - \mathbf{A^T(q)}\boldsymbol{\lambda},\tag{1}$$

where q ¼ q<sup>1</sup> q<sup>2</sup> … qn <sup>T</sup> is the vector of generalized coordinates of the system with n degrees of freedom, M qð Þ<sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>n</sup> is the matrix of inertia, Vmð Þ <sup>q</sup>; <sup>q</sup>\_ <sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>n</sup> is the matrix of Coriolis and centrifugal forces, Fmð Þ <sup>q</sup>\_ <sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>1</sup> is the vector that represents viscous friction, Gmð Þ <sup>q</sup> <sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>1</sup> is the vector of gravitational torques, <sup>τ</sup><sup>d</sup> <sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>1</sup> is the disturbance vector, <sup>τ</sup><sup>∈</sup> <sup>R</sup><sup>r</sup>�<sup>1</sup> is the vector of input torques, where <sup>r</sup> is the number of inputs, B qð Þ<sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>r</sup> is the input transformation matrix, <sup>λ</sup><sup>∈</sup> <sup>R</sup><sup>m</sup>�<sup>1</sup> is the vector that represents restriction forces, and A qð Þ<sup>∈</sup> <sup>R</sup><sup>m</sup>�<sup>n</sup> is the matrix associated to such restrictions. Two well known properties of such model are [9, 22]:


DDMR are surveillance [2], floor cleaning [3], industrial load transportation [4], autonomous

In the literature, most of the motion controllers for DDMR are based only on its kinematics. The main reasons for that are: (a) the kinematic model is simpler than the dynamic model, therefore the resulting controllers are less complex and simpler to tune; (b) the accuracy of the dynamic model depends on several parameters that might change or are difficult to measure, like the robot's mass and moment of inertia; and (c) dynamic controllers usually generate torque or voltage commands, while mobile robots frequently have internal velocity controllers that take velocity as input [6]. However, the robot's low-level velocity control loops do not guarantee perfect velocity tracking, especially when high-speed movements and/or heavy load transportation are required. In such cases, to reduce tracking error, it becomes essential to

A possible solution to overcome the problem described above is to design a controller that compensates for the robot's dynamics. Commercial mobile robots usually have internal controllers that accept velocity commands, like the Pioneer 3 from Adept Mobile Robots, the Khepera from K-Team Corporation, and the robuLAB-10 from Robosoft Inc. However, the control signals generated by most dynamic controllers in the literature are torques or voltages, as in [8–14]. Because of that, some researchers have proposed dynamic controllers that generate linear and angular velocities as commands [15, 16]. In some works, the dynamic model is divided in to two parts, allowing the design of independent controllers for the robot kinematics and dynamics [17–20]. Finally, to reduce performance degradation in applications in which the robot dynamic parameters may vary (such as load transportation) or when the knowledge of the dynamic parameters is imprecise, adaptive controllers can also be considered [7, 21].

The above-mentioned works applied a dynamic model that has linear and angular velocities as inputs, which illustrates the interest on such kind of dynamic model. In such context, this chapter explains the velocity-based dynamic model and its mathematical properties, which are useful for the design of controllers that compensate for the robot dynamics. It also illustrates how to design a trajectory tracking motion controller based on the robot's kinematics, and how

The classical equation to represent the dynamics of mobile robots can be obtained via Lagrang-

degrees of freedom, M qð Þ<sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>n</sup> is the matrix of inertia, Vmð Þ <sup>q</sup>; <sup>q</sup>\_ <sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>n</sup> is the matrix of Coriolis and centrifugal forces, Fmð Þ <sup>q</sup>\_ <sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>1</sup> is the vector that represents viscous friction, Gmð Þ <sup>q</sup> <sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>1</sup> is the vector of gravitational torques, <sup>τ</sup><sup>d</sup> <sup>∈</sup> <sup>R</sup><sup>n</sup>�<sup>1</sup> is the disturbance vector,

M qð Þq€ <sup>þ</sup> Vmð Þ <sup>q</sup>; <sup>q</sup>\_ <sup>q</sup>\_ <sup>þ</sup> Fmð Þþ <sup>q</sup>\_ Gmð Þþ <sup>q</sup> <sup>τ</sup>d¼B qð Þ<sup>τ</sup> � <sup>A</sup><sup>T</sup>ð Þ <sup>q</sup> <sup>λ</sup>, (1)

<sup>T</sup> is the vector of generalized coordinates of the system with n

to integrate it with a velocity-based dynamic compensation controller.

wheelchairs [5], and others.

24 Applications of Mobile Robots

2. Dynamic model

ian formulation, resulting in [22].

where q ¼ q<sup>1</sup> q<sup>2</sup> … qn

consider the robot dynamics as well, as shown in [7].

The above properties are widely used on the development and stability analysis of controllers for mobile robots, as shown in [8, 9, 22, 23]. But, such controllers generate torque commands, not velocities, as usually accepted by commercial robots. The conversion from torque to velocity commands requires knowledge of the actuation system of the robot (model of its motors and its speed controllers). On the other hand, a controller designed from a velocitybased dynamic model generates linear and angular velocities that can be directly applied as commands for mobile robots.

In such a context, now the dynamic model for the DDMR proposed in [16] is reviewed. For convenience, we first present its equations again. Then, the dynamic model is written in such a way that it becomes similar to the classical dynamic equation based on torques. Figure 1 depicts a DDMR with the variables of interest. There, u and ω are, respectively, the linear and angular velocities, G is the center of mass, h is the point of interest (whose position should be controlled)

Figure 1. The differential drive mobile robot (DDMR). u and ω are, respectively, the linear and angular velocities, G is the center of mass, h is the point of interest with coordinates x and y in the XY plane, ψ is the robot orientation, a is the distance from the point of interest to the point in the middle of the virtual axle that links the traction wheels (point B), b is the distance between points G and B, and d is the distance between the points of contact of the traction wheels to the floor.

with coordinates x and y in the XY plane, ψ is the robot orientation, a is the distance from the point of interest to the point in the middle of the virtual axle that links the traction wheels (point B), b is the distance between points G and B, and d is the distance between the points of contact of the traction wheels to the floor. The complete mathematical model is written as [16].

$$
\begin{bmatrix}
\dot{\boldsymbol{x}} \\
\dot{\boldsymbol{y}} \\
\dot{\boldsymbol{w}} \\
\dot{\boldsymbol{u}} \\
\dot{\boldsymbol{w}}
\end{bmatrix} = \begin{bmatrix}
u\cos\psi - a\omega\sin\psi \\
u\sin\psi + a\omega\cos\psi \\
\omega \\
\frac{\theta\_3}{\theta\_1}\omega^2 - \frac{\theta\_4}{\theta\_1}u \\
\end{bmatrix} + \begin{bmatrix}
0 & 0 \\
0 & 0 \\
\frac{1}{\theta\_1} & 0 \\
0 & 1 \\
0 & \frac{1}{\theta\_2}
\end{bmatrix} \begin{bmatrix}
\delta\_x \\
\delta\_y \\
0 \\
\delta\_u \\
\delta\_{\omega}
\end{bmatrix},
\tag{2}
$$

<sup>H</sup><sup>0</sup> <sup>¼</sup> <sup>θ</sup><sup>1</sup> <sup>0</sup> 0 θ<sup>2</sup>

=s), such that

The role of the term <sup>i</sup> <sup>¼</sup> <sup>1</sup>rad<sup>2</sup>

following matrices are defined:

<sup>H</sup> <sup>¼</sup> <sup>θ</sup>1=<sup>i</sup> <sup>0</sup> 0 θ<sup>2</sup>

or

ur ωr  <sup>¼</sup> <sup>θ</sup>1=<sup>i</sup> <sup>0</sup> 0 θ<sup>2</sup> iu\_

vector vr are kept unchanged.

so that the term c vð Þv can be written as

the vector of modified velocities, so that

<sup>i</sup> <sup>¼</sup> <sup>1</sup>rad<sup>2</sup>

, c vð Þ¼ <sup>θ</sup><sup>4</sup> �θ3<sup>ω</sup>

0 �θ3ω θ3ω 0 iu

, F v<sup>0</sup> ð Þ¼ <sup>θ</sup>4=<sup>i</sup> <sup>0</sup>

ω\_ 

Notice that c vð Þv ¼ C v<sup>0</sup> ð Þv<sup>0</sup> þ F v<sup>0</sup> ð Þv<sup>0</sup> and H<sup>0</sup>

þ

Finally, the dynamic model of a DDMR can be represented by

0 �θ3ω θ3ω 0 iu

θ5ω θ<sup>6</sup>

c vð Þ¼ <sup>θ</sup><sup>4</sup> �θ3<sup>ω</sup>

matrices, while keeping the numerical values unchanged. Now, let us define v<sup>0</sup> ¼ ½ � iu ω

<sup>v</sup><sup>0</sup> <sup>¼</sup> <sup>i</sup> <sup>0</sup> 0 1 <sup>u</sup>

The terms in the vector of modified velocities are numerically equal to the terms in the vector of actual velocities v, only its dimensions are different. By rewriting the model equation, the

> ω þ

> > v\_¼Hv\_ 0

The model represented by Eq. (13) is mathematically equivalent to the one proposed in [16] and used in [7], where it was validated via simulation and experiments. Nevertheless, the model presented here is written in such a way that some mathematical properties arise. Such

ω þ

Let us rewrite c vð Þ by adding and subtracting the term iθ3u to its fourth element (where

θ5ω θ<sup>6</sup> þ ð Þ iθ<sup>3</sup> � iθ<sup>3</sup> u

θ<sup>4</sup> 0

ω

0 θ<sup>6</sup> þ ð Þ θ<sup>5</sup> � iθ<sup>3</sup> u u

=s is to make the units consistent to allow us to split c vð Þ into two

<sup>0</sup> <sup>θ</sup><sup>6</sup> <sup>þ</sup> ð Þ <sup>θ</sup>5=<sup>i</sup> � <sup>θ</sup><sup>3</sup> iu , and C v<sup>0</sup> ð Þ¼ <sup>0</sup> �θ3<sup>ω</sup>

vr¼Hv\_<sup>0</sup> þ C v<sup>0</sup> ð Þv<sup>0</sup> þ F v<sup>0</sup> ð Þv<sup>0</sup> þ Δ, (12)

θ4=i 0

<sup>0</sup> <sup>θ</sup><sup>6</sup> <sup>þ</sup> ð Þ <sup>θ</sup>5=<sup>i</sup> � <sup>θ</sup><sup>3</sup> iu iu

, that is, the dimensions of the resulting

and <sup>Δ</sup> <sup>¼</sup> �θ<sup>1</sup> <sup>0</sup>

0 �θ<sup>2</sup> δ<sup>u</sup>

http://dx.doi.org/10.5772/intechopen.79397

, (8)

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

ω

: (10)

δω

: (9)

θ3ω 0 :

> ω þ δu δω :

T as

(11)

(13)

: (7)

27

where θ ¼ ½ � θ1;…; θ<sup>6</sup> <sup>T</sup> is the vector of identified parameters and <sup>δ</sup> <sup>¼</sup> <sup>δ</sup>x; <sup>δ</sup>y; <sup>0</sup>; <sup>δ</sup>u; δω � �<sup>T</sup> is the vector of parametric uncertainties associated to the mobile robot. The equations describing the parameters θ are presented in Section 3. The model is split into kinematic and dynamic parts. The kinematic model is

$$
\begin{bmatrix}
\dot{\mathbf{x}} \\
\dot{\boldsymbol{y}} \\
\dot{\boldsymbol{\psi}}
\end{bmatrix} = \begin{bmatrix}
\cos\psi & -a\sin\psi \\
\sin\psi & a\cos\psi \\
\mathbf{0} & 1
\end{bmatrix} \begin{bmatrix}
u \\
\omega \\
\end{bmatrix} + \begin{bmatrix}
\delta\_{\mathbf{x}} \\
\delta\_{\mathbf{y}} \\
\mathbf{0}
\end{bmatrix} \tag{3}
$$

which is a modified approach to describe the robot kinematics. The classical unicycle model is obtained when a ¼ 0 in (3), but here we consider the case in which a 6¼ 0, which means that the ð Þ x; y position described by the model is not in the center of the line between the traction wheels, but at a distance a from it (see point h in Figure 1). We use this model because it is useful on the design of the trajectory tracking controller, as shown in Section 4.

The part of the equation that represents the dynamics is given by

$$
\begin{bmatrix} \dot{u} \\ \dot{\omega} \end{bmatrix} = \begin{bmatrix} \frac{\Theta\_3}{\Theta\_1} \omega^2 - \frac{\Theta\_4}{\Theta\_1} u \\\\ -\frac{\Theta\_5}{\Theta\_2} u \omega - \frac{\Theta\_6}{\Theta\_2} \omega \end{bmatrix} + \begin{bmatrix} 1 & 0 \\ \frac{\Theta\_1}{\Theta\_1} & 0 \\ 0 & \frac{1}{\Theta\_2} \end{bmatrix} \begin{bmatrix} u\_r \\ \omega\_r \end{bmatrix} + \begin{bmatrix} \delta\_u \\ \delta\_\omega \end{bmatrix}.\tag{4}
$$

As shown in [21], by rearranging its terms the Eq. (4) can be written as

$$
\begin{bmatrix} -\theta\_1 & 0\\ 0 & -\theta\_2 \end{bmatrix} \begin{bmatrix} \delta\_u\\ \delta\_\omega \end{bmatrix} + \begin{bmatrix} \theta\_1 & 0\\ 0 & \theta\_2 \end{bmatrix} \begin{bmatrix} \dot{u}\\ \dot{w} \end{bmatrix} + \begin{bmatrix} \theta\_4 & -\theta\_3\omega\\ \theta\_5\omega & \theta\_6 \end{bmatrix} \begin{bmatrix} u\\ \omega \end{bmatrix} = \begin{bmatrix} 1 & 0\\ 0 & 1 \end{bmatrix} \begin{bmatrix} u\_r\\ \omega\_r \end{bmatrix},\tag{5}
$$

or, in a compact form, as

$$
\Delta + \mathbf{H}' \dot{\mathbf{v}} + \mathbf{c}(\mathbf{v}) \mathbf{v} = \mathbf{v}\_{\mathbf{r}} \tag{6}
$$

where vr <sup>¼</sup> ur <sup>ω</sup><sup>r</sup> ½ �<sup>T</sup> is the vector of reference velocities, <sup>v</sup> <sup>¼</sup> ½ � <sup>u</sup> <sup>ω</sup> <sup>T</sup> is the vector containing the actual robot velocities, and the matrices H<sup>0</sup> and c vð Þ, and the vector Δ are given by

$$\mathbf{H}' = \begin{bmatrix} \theta\_1 & 0\\ 0 & \theta\_2 \end{bmatrix}, \qquad \mathbf{c}(\mathbf{v}) = \begin{bmatrix} \theta\_4 & -\theta\_3 \omega\\ \theta\_5 \omega & \theta\_6 \end{bmatrix} \quad \text{and} \quad \boldsymbol{\Delta} = \begin{bmatrix} -\theta\_1 & 0\\ 0 & -\theta\_2 \end{bmatrix} \begin{bmatrix} \delta\_u\\ \delta\_w \end{bmatrix}. \tag{7}$$

Let us rewrite c vð Þ by adding and subtracting the term iθ3u to its fourth element (where <sup>i</sup> <sup>¼</sup> <sup>1</sup>rad<sup>2</sup> =s), such that

$$\mathbf{c}(\mathbf{v}) = \begin{bmatrix} \theta\_4 & -\theta\_3 \omega \\ \theta\_5 \omega & \theta\_6 + (i\theta\_3 - i\theta\_3)\mu \end{bmatrix}' \tag{8}$$

so that the term c vð Þv can be written as

$$
\begin{bmatrix} 0 & -\theta\_3 \omega \\ \theta\_3 \omega & 0 \end{bmatrix} \begin{bmatrix} \dot{u} \\ \omega \end{bmatrix} + \begin{bmatrix} \theta\_4 & 0 \\ 0 & \theta\_6 + (\theta\_5 - i\theta\_3)u \end{bmatrix} \begin{bmatrix} u \\ \omega \end{bmatrix}. \tag{9}
$$

The role of the term <sup>i</sup> <sup>¼</sup> <sup>1</sup>rad<sup>2</sup> =s is to make the units consistent to allow us to split c vð Þ into two matrices, while keeping the numerical values unchanged. Now, let us define v<sup>0</sup> ¼ ½ � iu ω T as the vector of modified velocities, so that

$$\mathbf{v}' = \begin{bmatrix} i & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} u \\ w \end{bmatrix}. \tag{10}$$

The terms in the vector of modified velocities are numerically equal to the terms in the vector of actual velocities v, only its dimensions are different. By rewriting the model equation, the following matrices are defined:

$$\mathbf{H} = \begin{bmatrix} \theta\_1/\text{i} & 0\\ 0 & \theta\_2 \end{bmatrix}, \qquad \mathbf{F}(\mathbf{v}') = \begin{bmatrix} \theta\_4/\text{i} & 0\\ 0 & \theta\_6 + (\theta\_5/\text{i} - \theta\_3)\text{i}u \end{bmatrix}, \quad \text{and} \quad \mathbf{C}(\mathbf{v}') = \begin{bmatrix} 0 & -\theta\_3\omega\\ \theta\_3\omega & 0 \end{bmatrix}. \tag{11}$$

Finally, the dynamic model of a DDMR can be represented by

$$\mathbf{v\_{r}=H\dot{\mathbf{v}}^{\prime}+\mathbf{C}(\mathbf{v}^{\prime})\mathbf{v}^{\prime}+\mathbf{F}(\mathbf{v}^{\prime})\mathbf{v}^{\prime}+\boldsymbol{\Delta}},\tag{12}$$

or

with coordinates x and y in the XY plane, ψ is the robot orientation, a is the distance from the point of interest to the point in the middle of the virtual axle that links the traction wheels (point B), b is the distance between points G and B, and d is the distance between the points of contact of the traction wheels to the floor. The complete mathematical model is written as [16].

þ

vector of parametric uncertainties associated to the mobile robot. The equations describing the parameters θ are presented in Section 3. The model is split into kinematic and dynamic parts.

which is a modified approach to describe the robot kinematics. The classical unicycle model is obtained when a ¼ 0 in (3), but here we consider the case in which a 6¼ 0, which means that the ð Þ x; y position described by the model is not in the center of the line between the traction wheels, but at a distance a from it (see point h in Figure 1). We use this model because it is

ω\_ � � þ

the actual robot velocities, and the matrices H<sup>0</sup> and c vð Þ, and the vector Δ are given by

Δ þ H<sup>0</sup>

1 θ1

0

θ<sup>4</sup> �θ3ω θ5ω θ<sup>6</sup> � � u

ur ωr � � þ δu δω � �

ω � �

v\_ þ c vð Þv¼vr, (6)

<sup>¼</sup> 1 0 0 1 � � ur

<sup>0</sup> <sup>1</sup> θ2

cos ψ �a sinψ sin ψ a cosψ 0 1

<sup>0</sup> <sup>1</sup> θ2

<sup>T</sup> is the vector of identified parameters and <sup>δ</sup> <sup>¼</sup> <sup>δ</sup>x; <sup>δ</sup>y; <sup>0</sup>; <sup>δ</sup>u; δω

3 7 5 u ω � � þ

0

ur ωr � �

> δx δy 0

3 7

2 6 4 þ

δx δy 0 δu δω

, (2)

� �<sup>T</sup> is the

<sup>5</sup>, (3)

: (4)

ωr � �

<sup>T</sup> is the vector containing

, (5)

u cos ψ � aω sinψ u sin ψ þ aω cosψ ω

> <sup>ω</sup><sup>2</sup> � <sup>θ</sup><sup>4</sup> θ1 u

<sup>u</sup><sup>ω</sup> � <sup>θ</sup><sup>6</sup> θ2 ω

θ3 θ1

� θ5 θ2

x\_ y\_ ψ\_

2 6 4

useful on the design of the trajectory tracking controller, as shown in Section 4.

<sup>ω</sup><sup>2</sup> � <sup>θ</sup><sup>4</sup> θ1 u

<sup>u</sup><sup>ω</sup> � <sup>θ</sup><sup>6</sup> θ2 ω

As shown in [21], by rearranging its terms the Eq. (4) can be written as

θ<sup>1</sup> 0 0 θ<sup>2</sup> � � u\_

where vr <sup>¼</sup> ur <sup>ω</sup><sup>r</sup> ½ �<sup>T</sup> is the vector of reference velocities, <sup>v</sup> <sup>¼</sup> ½ � <sup>u</sup> <sup>ω</sup>

The part of the equation that represents the dynamics is given by

θ3 θ1

� θ5 θ2

u\_ ω\_ � �

�θ<sup>1</sup> 0 0 �θ<sup>2</sup> � � δ<sup>u</sup>

or, in a compact form, as

¼

δω � �

þ

2 6 4

x\_ y\_ ψ\_ u\_ ω\_

where θ ¼ ½ � θ1;…; θ<sup>6</sup>

26 Applications of Mobile Robots

The kinematic model is

$$
\begin{bmatrix} u\_r \\ \omega\_r \end{bmatrix} = \begin{bmatrix} \theta\_1/i & 0 \\ 0 & \theta\_2 \end{bmatrix} \begin{bmatrix} \dot{u} \\ \dot{\omega} \end{bmatrix} + \begin{bmatrix} 0 & -\theta\_3\omega \\ \theta\_3\omega & 0 \end{bmatrix} \begin{bmatrix} \dot{u} \\ \omega \end{bmatrix} + \begin{bmatrix} \theta\_4/i & 0 \\ 0 & \theta\_6 + (\theta\_5/i - \theta\_3)\dot{u} \end{bmatrix} \begin{bmatrix} \dot{u} \\ \omega \end{bmatrix} + \begin{bmatrix} \delta\_u \\ \delta\_\omega \end{bmatrix}.\tag{13}
$$

Notice that c vð Þv ¼ C v<sup>0</sup> ð Þv<sup>0</sup> þ F v<sup>0</sup> ð Þv<sup>0</sup> and H<sup>0</sup> v\_¼Hv\_ 0 , that is, the dimensions of the resulting vector vr are kept unchanged.

The model represented by Eq. (13) is mathematically equivalent to the one proposed in [16] and used in [7], where it was validated via simulation and experiments. Nevertheless, the model presented here is written in such a way that some mathematical properties arise. Such properties, which are presented and discussed in the next session, can be applied on the design and stability analysis of dynamic controllers.

i ¼ 1; 2; 4; 6. The parameters θ<sup>3</sup> and θ<sup>5</sup> can be negative and will be null if, and only if, the center of mass G is exactly in the center of the virtual axle, that is, b ¼ 0. Finally, in [21], it was shown that the model parameters θ<sup>1</sup> to θ<sup>6</sup> cannot be written as a linear combination of each other, that

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

http://dx.doi.org/10.5772/intechopen.79397

29

3. The matrix F v<sup>0</sup> ð Þ is symmetric and positive definite, or F v<sup>0</sup> ð Þ¼ <sup>F</sup><sup>T</sup> <sup>&</sup>gt; 0, if <sup>θ</sup><sup>6</sup> <sup>&</sup>gt; �ð Þ <sup>θ</sup>5=<sup>i</sup> � <sup>θ</sup><sup>3</sup> iu;

6. The matrix F v<sup>0</sup> ð Þ can be considered constant if θ<sup>6</sup> ≫ ∣ð Þ θ5=i � θ<sup>3</sup> iu∣ and there is no change

To analyze the above mathematical properties, first recall that θ<sup>i</sup> > 0 for i ¼ 1; 2; 4; 6. Properties 1 and 2 can be confirmed by observing that H is a diagonal square matrix formed by θ<sup>1</sup> and θ2. F v<sup>0</sup> ð Þ is also a diagonal square matrix formed by θ<sup>4</sup> and θ<sup>6</sup> þ ð Þ θ5=i � θ<sup>3</sup> iu. Property 3 holds if θ<sup>6</sup> > �ð Þ θ5=i � θ<sup>3</sup> iu. Property 4 holds if there is no change on the physical parameters of the robot (i.e., if there is no change on the robot's mass, moment of inertia, etc.). C v<sup>0</sup> ð Þ is a square matrix formed by θ3ω and �θ3ω, whose transpose is also its negative, which proves property 5. Property 6 holds if there is no change on the physical parameters of the robot and

The values of the dynamic parameters θ can be estimated via an identification procedure,

where θ is the vector of parameters and Y is the system output. The least squares estimate of θ

where θb is the vector with the estimated values of θ and W is the regression matrix. By

<sup>θ</sup><sup>b</sup> <sup>¼</sup> <sup>W</sup>TW � ��<sup>1</sup>

rearranging (4) and ignoring uncertainty, the dynamic model can be represented by

Y¼Wθ, (15)

WTY, (16)

4. The matrix H is constant if there is no change on the physical parameters of the robot;

7. The mapping vr ! v<sup>0</sup> is strictly output passive if θ<sup>6</sup> > �ð Þ θ5=i � θ<sup>3</sup> iu and Δ ¼ 0.

is, they are independent.

3.1. Model properties

The mathematical properties of the dynamic model (12) are:

5. The matrix C v<sup>0</sup> ð Þ is skew symmetric;

3.2. Identified parameters

is given by

on the physical parameters of the robot;

1. The matrix <sup>H</sup> is symmetric and positive definite, or <sup>H</sup> <sup>¼</sup> <sup>H</sup><sup>T</sup> <sup>&</sup>gt; 0; 2. The inverse of H exists and is also positive definite, or ∃ H�<sup>1</sup> > 0;

θ<sup>6</sup> ≫ ∣ð Þ θ5=i � θ<sup>3</sup> iu∣. Finally, the proof for property 7 is given in [21].

described as follows. Let a system be represented by the regression model

#### 3. Dynamic parameters and model properties

To calculate the dynamic parameters of the vector θ, one has to know physical parameters of the robot, like its mass, its moment of inertia, friction coefficient of its motors, etc. The equations describing each one of the parameters θ<sup>i</sup> are

$$\begin{aligned} \Theta\_1 &= \left[ \frac{R\_a}{k\_a} \left( mr^2 + 2l\_\epsilon \right) + 2rk\_{D\Gamma} \right] \frac{1}{\left( 2r k\_{P\Gamma} \right)} \quad [\text{s]}, \\ \Theta\_2 &= \left[ \frac{R\_a}{k\_a} \left( l\_\epsilon d^2 + 2r^2 \left( l\_z + mb^2 \right) \right) + 2rdk\_{D\Gamma} \right] \frac{1}{\left( 2r dk\_{P\Gamma} \right)} \quad [\text{s]}, \\ \Theta\_3 &= \frac{R\_a}{k\_a} \frac{mbr}{2k\_{P\Gamma}} \quad [sm/rad^2], \\ \Theta\_4 &= \frac{R\_a}{k\_a} \left( \frac{k\_a k\_b}{R\_a} + B\_\epsilon \right) \frac{1}{r k\_{P\Gamma}} + 1, \\ \Theta\_5 &= \frac{R\_a}{k\_a} \frac{mbr}{dk\_{P\Gamma}} \quad [\text{s/}m], \text{ and} \\ \Theta\_6 &= \frac{R\_a}{k\_a} \left( \frac{k\_a k\_b}{R\_a} + B\_\epsilon \right) \frac{d}{2rk\_{P\Gamma}} + 1, \end{aligned} \tag{14}$$

where m is the mass of the robot, Iz is its moment of inertia at G, Ra, kb and ka are the electrical resistance, the electromotive constant, and the constant of torque of its motors, respectively, Be is the coefficient of friction, Ie is the moment of inertia of each group rotor-reduction gearwheel, r is the radius of each wheel, and b and d are distances defined in Figure 1. It is assumed that the internal motor controllers are of type PD (proportional-derivative) with proportional gains kPT > 0 and kPR > 0, and derivative gains kDT ≥ 0 and kDR ≥ 0. It is also assumed that the inductances of the motors are negligible, and both driving motors are identical.

Obtaining accurate values of all physical parameters of a robot might be difficult, or even not possible. Therefore, it is useful to discuss an identification procedure to directly obtain the values of the dynamic parameters θ. Such procedure is explained in Section 3.2.

It is interesting to point out that the dynamic model adopted here considers that the robot's center of mass G can be located anywhere along the line that crosses the center of the structure, as illustrated in Figure 1. This means that the formulation of the proposed dynamic model is adequate for robots that have a symmetrical weight distribution between their left and right sides. Because most differential drive robots have an approximately symmetrical weight distribution (with each motor and wheel on either left or right sides), such assumption does not introduce significant modeling errors on most cases. It should also be noticed that θ<sup>i</sup> > 0 for i ¼ 1; 2; 4; 6. The parameters θ<sup>3</sup> and θ<sup>5</sup> can be negative and will be null if, and only if, the center of mass G is exactly in the center of the virtual axle, that is, b ¼ 0. Finally, in [21], it was shown that the model parameters θ<sup>1</sup> to θ<sup>6</sup> cannot be written as a linear combination of each other, that is, they are independent.

#### 3.1. Model properties

properties, which are presented and discussed in the next session, can be applied on the design

To calculate the dynamic parameters of the vector θ, one has to know physical parameters of the robot, like its mass, its moment of inertia, friction coefficient of its motors, etc. The

> <sup>2</sup> Iz <sup>þ</sup> mb<sup>2</sup> <sup>þ</sup> <sup>2</sup>rdkDR 1

> > sm=rad<sup>2</sup> ,

rkPT þ 1,

2rkPR

þ 1,

where m is the mass of the robot, Iz is its moment of inertia at G, Ra, kb and ka are the electrical resistance, the electromotive constant, and the constant of torque of its motors, respectively, Be is the coefficient of friction, Ie is the moment of inertia of each group rotor-reduction gearwheel, r is the radius of each wheel, and b and d are distances defined in Figure 1. It is assumed that the internal motor controllers are of type PD (proportional-derivative) with proportional gains kPT > 0 and kPR > 0, and derivative gains kDT ≥ 0 and kDR ≥ 0. It is also assumed that the

Obtaining accurate values of all physical parameters of a robot might be difficult, or even not possible. Therefore, it is useful to discuss an identification procedure to directly obtain the

It is interesting to point out that the dynamic model adopted here considers that the robot's center of mass G can be located anywhere along the line that crosses the center of the structure, as illustrated in Figure 1. This means that the formulation of the proposed dynamic model is adequate for robots that have a symmetrical weight distribution between their left and right sides. Because most differential drive robots have an approximately symmetrical weight distribution (with each motor and wheel on either left or right sides), such assumption does not introduce significant modeling errors on most cases. It should also be noticed that θ<sup>i</sup> > 0 for

½ � s=m , and

inductances of the motors are negligible, and both driving motors are identical.

values of the dynamic parameters θ. Such procedure is explained in Section 3.2.

ð Þ 2rkPT

½ �s ,

ð Þ 2rdkPR

½ �s ,

(14)

and stability analysis of dynamic controllers.

28 Applications of Mobile Robots

3. Dynamic parameters and model properties

equations describing each one of the parameters θ<sup>i</sup> are

mr<sup>2</sup> <sup>þ</sup> <sup>2</sup>Ie <sup>þ</sup> <sup>2</sup>rkDT 1

Ied<sup>2</sup> <sup>þ</sup> <sup>2</sup><sup>r</sup>

mbr 2kPT

kakb Ra þ Be 1

mbr dkPR

kakb Ra þ Be d

<sup>θ</sup><sup>1</sup> <sup>¼</sup> Ra ka

<sup>θ</sup><sup>2</sup> <sup>¼</sup> Ra ka

<sup>θ</sup><sup>3</sup> <sup>¼</sup> Ra ka

<sup>θ</sup><sup>4</sup> <sup>¼</sup> Ra ka

<sup>θ</sup><sup>5</sup> <sup>¼</sup> Ra ka

<sup>θ</sup><sup>6</sup> <sup>¼</sup> Ra ka

The mathematical properties of the dynamic model (12) are:


To analyze the above mathematical properties, first recall that θ<sup>i</sup> > 0 for i ¼ 1; 2; 4; 6. Properties 1 and 2 can be confirmed by observing that H is a diagonal square matrix formed by θ<sup>1</sup> and θ2. F v<sup>0</sup> ð Þ is also a diagonal square matrix formed by θ<sup>4</sup> and θ<sup>6</sup> þ ð Þ θ5=i � θ<sup>3</sup> iu. Property 3 holds if θ<sup>6</sup> > �ð Þ θ5=i � θ<sup>3</sup> iu. Property 4 holds if there is no change on the physical parameters of the robot (i.e., if there is no change on the robot's mass, moment of inertia, etc.). C v<sup>0</sup> ð Þ is a square matrix formed by θ3ω and �θ3ω, whose transpose is also its negative, which proves property 5. Property 6 holds if there is no change on the physical parameters of the robot and θ<sup>6</sup> ≫ ∣ð Þ θ5=i � θ<sup>3</sup> iu∣. Finally, the proof for property 7 is given in [21].

#### 3.2. Identified parameters

The values of the dynamic parameters θ can be estimated via an identification procedure, described as follows. Let a system be represented by the regression model

$$\mathbf{Y} = \mathbf{W}\boldsymbol{\theta},\tag{15}$$

where θ is the vector of parameters and Y is the system output. The least squares estimate of θ is given by

$$
\widehat{\boldsymbol{\theta}} = \left(\mathbf{W}^{\mathsf{T}}\mathbf{W}\right)^{-1}\mathbf{W}^{\mathsf{T}}\mathbf{Y},\tag{16}
$$

where θb is the vector with the estimated values of θ and W is the regression matrix. By rearranging (4) and ignoring uncertainty, the dynamic model can be represented by

$$
\begin{bmatrix} u\_r \\ \omega\_r \end{bmatrix} = \begin{bmatrix} \dot{u} & 0 & -\omega^2 & u & 0 & 0 \\ 0 & \dot{\omega} & 0 & 0 & u\omega & \omega \end{bmatrix} \begin{bmatrix} \theta\_1 & \theta\_2 \ \theta\_3 \ \theta\_4 \ \theta\_5 \ \theta\_6 \end{bmatrix}^T \tag{17}
$$

can verify that the conditions of θ<sup>6</sup> > �ð Þ θ5=i � θ<sup>3</sup> iu and θ<sup>6</sup> ≫ ∣ð Þ θ5=i � θ<sup>3</sup> iu∣ are valid for all sets of identified parameters. Therefore, the dynamic model of the above-mentioned robots can

To illustrate the usefulness of the modified model and its properties, in this section we show the design of a trajectory tracking controller and a dynamic compensation controller. The controller design is split in two parts, as in [7]. The first part is based on the inverse kinematics and the second one compensates for the robot dynamics. The use of the dynamic model

The control structure is shown in Figure 2, where blocks K, D, and R represent the kinematic controller, the dynamic compensation controller, and the robot, respectively. Figure 2 shows

<sup>d</sup> from the trajectory planner (which is not considered in this work). Then, based on those

values and the estimates of the robot parameters θ to generate the velocity commands

The same kinematic controller presented in [7, 21] is shown here. It is a trajectory tracking controller based on the inverse kinematics of the robot. If only the position of the point of

<sup>T</sup> is considered, the robot's inverse kinematics can be written as

cosψ sinψ

sin<sup>ψ</sup> <sup>1</sup> a cosψ

Figure 2. Structure of the control system. The kinematic controller K receives the desired values of position hd and

<sup>d</sup>, the actual robot position h and its orientation ψ, and calculates the desired robot velocities vd. Those values and the actual robot velocities v are fed into the dynamic compensation controller D, that generates the velocity

� 1 a

2 4 � �<sup>T</sup> and velocity

<sup>T</sup> and orientation ψ, the kinematic controller

<sup>T</sup> are fed into the dynamic controller. Such controller uses those

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

http://dx.doi.org/10.5772/intechopen.79397

31

3 5 x\_ y\_

<sup>T</sup>. The desired velocities vd and the actual

� �: (19)

that the kinematic controller receives the desired values of position hd ¼ xd yd

vr <sup>¼</sup> ur <sup>ω</sup><sup>r</sup> ½ �<sup>T</sup> that are sent as references to the robot internal controller.

u ω � � ¼

be represented as in (12), with properties 1–7 valid under the considered conditions.

4. Controller design

robot velocities v ¼ ½ � u ω

4.1. Kinematic controller

interest h ¼ ½ � x y

velocity h\_

commands vr that are sent as references to the robot R.

h\_

properties is shown on the second part.

values and on the actual robot position h ¼ ½ � x y

calculates the desired robot velocities vd ¼ ½ � ud ω<sup>d</sup>

where

$$\mathbf{Y} = \begin{bmatrix} u\_r \\ \omega\_r \end{bmatrix}, \mathbf{W} = \begin{bmatrix} \dot{u} & 0 & -\omega^2 & u & 0 & 0 \\ 0 & \dot{\omega} & 0 & 0 & u\omega & \omega \end{bmatrix}, \boldsymbol{\theta} = \begin{bmatrix} \boldsymbol{\theta}\_1 & \boldsymbol{\theta}\_2 & \boldsymbol{\theta}\_3 & \boldsymbol{\theta}\_4 & \boldsymbol{\theta}\_5 & \boldsymbol{\theta}\_6 \end{bmatrix}^T. \tag{18}$$

In order to obtain an estimate for the values of θ, each robot needs to be excited with speed reference signals ur ð Þ ; ω<sup>r</sup> , while the actual values of its velocities ð Þ u; ω and accelerations ð Þ u\_; ω\_ are measured and stored. In our case, the excitation signals consisted of a sum of six sine waves with different frequencies and amplitudes. All data were stored and the regression model was assembled so that the vector Y and the matrix W had all values obtained in each sampling instant. Subsequently, the value of θ for each robot was calculated by least squares method.

In order to verify the assumptions that θ<sup>6</sup> ≫ ∣ð Þ θ5=i � θ<sup>3</sup> iu∣ and θ<sup>6</sup> > �ð Þ θ5=i � θ<sup>3</sup> iu, we have analyzed the dynamic parameters of five differential drive robots obtained via identification procedure. The analysis was done considering the parameters of the following robots: a Pioneer 3-DX with no extra equipment (P3), a Pioneer 3-DX with a LASER scanner and omnidirectional camera (P3laser), a robotic wheelchair while carrying a 55 kg person (RW55), a robotic wheelchair while carrying a 125kg person (RW125), and a Khepera III (KIII). The Khepera III robot weighs 690 g, has a diameter of 13 cm and is 7 cm high. Its dynamic parameters were identified by Laut and were originally presented in [24]. By its turn, the Pioneer robots weigh about 9 kg, are 44 cm long, 38 cm wide, and 22 cm tall (without the LASER scanner). The LASER scanner weighs about 50% of the original robot weight, which produces an important change in the mass and moment of inertia of the structure. Finally, the robotic wheelchair presents an even greater difference in dynamics because of its own weight (about 70 kg) and the weight of the person that it is carrying. The dynamic parameters for the above-mentioned robots are presented in Table 1.



Table 1. Identified dynamic parameters of a Pioneer 3-DX with no extra equipment (P3), a Pioneer 3-DX with a LASER scanner (P3laser), a robotic wheelchair while carrying a 55 kg person (RW55), a robotic wheelchair while carrying a 125 kg person (RW125), and a Khepera III (KIII).

can verify that the conditions of θ<sup>6</sup> > �ð Þ θ5=i � θ<sup>3</sup> iu and θ<sup>6</sup> ≫ ∣ð Þ θ5=i � θ<sup>3</sup> iu∣ are valid for all sets of identified parameters. Therefore, the dynamic model of the above-mentioned robots can be represented as in (12), with properties 1–7 valid under the considered conditions.

#### 4. Controller design

ur ωr 

where

in Table 1.

person (RW125), and a Khepera III (KIII).

<sup>Y</sup> <sup>¼</sup> ur ωr 

30 Applications of Mobile Robots

<sup>¼</sup> <sup>u</sup>\_ <sup>0</sup> �ω<sup>2</sup> <sup>u</sup> 0 0 0 ω\_ 0 0 uω ω 

In order to obtain an estimate for the values of θ, each robot needs to be excited with speed reference signals ur ð Þ ; ω<sup>r</sup> , while the actual values of its velocities ð Þ u; ω and accelerations ð Þ u\_; ω\_ are measured and stored. In our case, the excitation signals consisted of a sum of six sine waves with different frequencies and amplitudes. All data were stored and the regression model was assembled so that the vector Y and the matrix W had all values obtained in each sampling instant. Subsequently, the value of θ for each robot was calculated by least squares method.

In order to verify the assumptions that θ<sup>6</sup> ≫ ∣ð Þ θ5=i � θ<sup>3</sup> iu∣ and θ<sup>6</sup> > �ð Þ θ5=i � θ<sup>3</sup> iu, we have analyzed the dynamic parameters of five differential drive robots obtained via identification procedure. The analysis was done considering the parameters of the following robots: a Pioneer 3-DX with no extra equipment (P3), a Pioneer 3-DX with a LASER scanner and omnidirectional camera (P3laser), a robotic wheelchair while carrying a 55 kg person (RW55), a robotic wheelchair while carrying a 125kg person (RW125), and a Khepera III (KIII). The Khepera III robot weighs 690 g, has a diameter of 13 cm and is 7 cm high. Its dynamic parameters were identified by Laut and were originally presented in [24]. By its turn, the Pioneer robots weigh about 9 kg, are 44 cm long, 38 cm wide, and 22 cm tall (without the LASER scanner). The LASER scanner weighs about 50% of the original robot weight, which produces an important change in the mass and moment of inertia of the structure. Finally, the robotic wheelchair presents an even greater difference in dynamics because of its own weight (about 70 kg) and the weight of the person that it is carrying. The dynamic parameters for the above-mentioned robots are presented

The value of u is limited to 0:5 m=s for the Khepera III robots, to 1:2 m=s for the Pioneer robots, and to 1:5 m=s for the robotic wheelchair. Therefore, using the values presented in Table 1 one

θ1½ �s 0.5338 0.2604 0.3759 0.4263 0.0228 θ2½ �s 0.2168 0.2509 0.0188 0.0289 0.0568 θ<sup>3</sup> sm=rad<sup>2</sup> �0.0134 �0.0005 0.0128 0.0058 �0.0001 θ<sup>4</sup> 0.9560 0.9965 1.0027 0.9883 1.0030 θ5½ � s=m �0.0843 0.0026 �0.0015 0.0134 0.0732 θ<sup>6</sup> 1.0590 1.0768 0.9808 0.9931 0.9981

Table 1. Identified dynamic parameters of a Pioneer 3-DX with no extra equipment (P3), a Pioneer 3-DX with a LASER scanner (P3laser), a robotic wheelchair while carrying a 55 kg person (RW55), a robotic wheelchair while carrying a 125 kg

P3 P3laser RW<sup>55</sup> RW<sup>125</sup> KIII

,<sup>W</sup> <sup>¼</sup> <sup>u</sup>\_ <sup>0</sup> �ω<sup>2</sup> <sup>u</sup> 0 0 0 ω\_ 0 0 uω ω  ½ � θ<sup>1</sup> θ<sup>2</sup> θ<sup>3</sup> θ<sup>4</sup> θ<sup>5</sup> θ<sup>6</sup>

, θ ¼ ½ � θ<sup>1</sup> θ<sup>2</sup> θ<sup>3</sup> θ<sup>4</sup> θ<sup>5</sup> θ<sup>6</sup>

T, (17)

<sup>T</sup>: (18)

To illustrate the usefulness of the modified model and its properties, in this section we show the design of a trajectory tracking controller and a dynamic compensation controller. The controller design is split in two parts, as in [7]. The first part is based on the inverse kinematics and the second one compensates for the robot dynamics. The use of the dynamic model properties is shown on the second part.

The control structure is shown in Figure 2, where blocks K, D, and R represent the kinematic controller, the dynamic compensation controller, and the robot, respectively. Figure 2 shows that the kinematic controller receives the desired values of position hd ¼ xd yd � �<sup>T</sup> and velocity h\_ <sup>d</sup> from the trajectory planner (which is not considered in this work). Then, based on those values and on the actual robot position h ¼ ½ � x y <sup>T</sup> and orientation ψ, the kinematic controller calculates the desired robot velocities vd ¼ ½ � ud ω<sup>d</sup> <sup>T</sup>. The desired velocities vd and the actual robot velocities v ¼ ½ � u ω <sup>T</sup> are fed into the dynamic controller. Such controller uses those values and the estimates of the robot parameters θ to generate the velocity commands vr <sup>¼</sup> ur <sup>ω</sup><sup>r</sup> ½ �<sup>T</sup> that are sent as references to the robot internal controller.

#### 4.1. Kinematic controller

The same kinematic controller presented in [7, 21] is shown here. It is a trajectory tracking controller based on the inverse kinematics of the robot. If only the position of the point of interest h ¼ ½ � x y <sup>T</sup> is considered, the robot's inverse kinematics can be written as

Figure 2. Structure of the control system. The kinematic controller K receives the desired values of position hd and velocity h\_ <sup>d</sup>, the actual robot position h and its orientation ψ, and calculates the desired robot velocities vd. Those values and the actual robot velocities v are fed into the dynamic compensation controller D, that generates the velocity commands vr that are sent as references to the robot R.

The inverse kinematics described by Eq. (19) is valid only for a 6¼ 0. This is the reason why we prefer to adopt this model instead of the classical unicycle model, as discussed earlier. Considering (19), the adopted control law is

$$
\begin{bmatrix} u\_d \\ \omega\_d \end{bmatrix} = \begin{bmatrix} \cos\psi & \sin\psi \\ -\frac{1}{a}\sin\psi & \frac{1}{a}\cos\psi \end{bmatrix} \begin{bmatrix} \dot{x}\_d + l\_x \tanh\left(\frac{k\_x}{l\_x}\ddot{x}\right) \\\\ \dot{y}\_d + l\_y \tanh\left(\frac{k\_y}{l\_y}\ddot{y}\right) \end{bmatrix}' \tag{20}
$$

~x tðÞ! 0 and ~y tðÞ! 0 as t ! ∞. This result will be revisited latter, after adding a dynamic

Remark. Considering the case in which the reference is a fixed destination point, instead of a trajectory, the robot reaches such a point and stops there. Assuming u � ud and ω � ωd,

Now, the use of the proposed dynamic model and its properties is illustrated via the design of a dynamic compensation controller. It receives the desired velocities vd from the kinematic controller and generates a pair of linear and angular velocity references vr for the robot servos,

> <sup>¼</sup> <sup>i</sup> <sup>0</sup> 0 1 � � ud

¼v<sup>0</sup> <sup>d</sup> � v<sup>0</sup> .

Regarding parametric uncertainties, the proposed dynamic compensation control law is

<sup>d</sup> <sup>þ</sup> <sup>T</sup> <sup>v</sup>~<sup>0</sup> ð Þ � � <sup>þ</sup> Cv<sup>b</sup> <sup>0</sup>

where Hb , Cb, and bF are estimates of H, C, and F, respectively, T v~<sup>0</sup> ð Þ¼

tion constants, and ω~ ¼ ω<sup>d</sup> � ω and u~ ¼ ud � u are the current velocity errors. The term T v~<sup>0</sup> ð Þ provides a saturation in order to guarantee that the commands to be sent to the robot are

In this chapter, we consider that the dynamic parameters are exactly known, that is, θb ¼ θ. This means that Hb ¼ H, Cb ¼ C, and bF ¼ F. The analysis considering parametric error is

HT <sup>v</sup>~<sup>0</sup> ð Þ� <sup>v</sup>~0<sup>T</sup>

Observing property 5, of antisymmetry of C, the derivative of the Lyapunov function can be

HT <sup>v</sup>~<sup>0</sup> ð Þ� <sup>v</sup>~0<sup>T</sup>

Cv~<sup>0</sup> � <sup>v</sup>~0<sup>T</sup>

Fv~<sup>0</sup>

Fv~<sup>0</sup>

ωd

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

<sup>d</sup> þ bFv<sup>0</sup>

, ku > 0 and k<sup>ω</sup> > 0 are gain constants, lu ∈ R and l<sup>ω</sup> ∈ R are satura-

<sup>d</sup> as

http://dx.doi.org/10.5772/intechopen.79397

33

<sup>d</sup>, (25)

<sup>d</sup> and v\_<sup>0</sup>

<sup>2</sup> <sup>v</sup>~0THv~<sup>0</sup> <sup>&</sup>gt; 0, and considering that the dynamic

: (26)

: (27)

<sup>d</sup> are bounded to

� �, (24)

controller to the system in order to implement the whole control scheme.

4.2. Dynamic compensation controller

and the vector of velocity errors is given by v~<sup>0</sup>

Using the Lyapunov candidate function <sup>V</sup> <sup>¼</sup> <sup>1</sup>

parameters are constant, one has

lu iu~ � �

tanh <sup>k</sup><sup>ω</sup> lω ω~ � �

lu 0 0 l<sup>ω</sup>

� � tanh ku

appropriate values.

presented in [7, 21].

written as

Eq. (20) guarantees that ω ¼ 0 when ~x ¼ 0 and ~y ¼ 0, therefore ψðÞ!t ψconstant.

as shown in Figure 2. First, let us define the vector of modified velocities v<sup>0</sup>

v0 <sup>d</sup> <sup>¼</sup> <sup>u</sup><sup>0</sup> d ωd � �

vr¼H<sup>b</sup> <sup>v</sup>\_<sup>0</sup>

always below the corresponding physical limits, considering that v<sup>0</sup>

<sup>V</sup>\_ ¼ �v~0<sup>T</sup>

<sup>V</sup>\_ ¼ �v~0<sup>T</sup>

for which vd ¼ ½ � ud ω<sup>d</sup> <sup>T</sup> is the vector of desired velocities given by the kinematic controller; h ¼ ½ � x y <sup>T</sup> and hd <sup>¼</sup> xd yd � �<sup>T</sup> are the vectors of actual and desired coordinates of the point of interest <sup>h</sup>, respectively; <sup>h</sup><sup>~</sup> <sup>¼</sup> ½ � <sup>~</sup><sup>x</sup> <sup>~</sup><sup>y</sup> <sup>T</sup> is the vector of position errors given by hd � <sup>h</sup>; kx <sup>&</sup>gt; 0 and ky > 0 are the controller gains; lx, ly ∈ R are saturation constants; and a > 0. The tanh terms are included to limit the values of the desired velocities vd to avoid saturation of the robot actuators in case the position errors h~ are too big, considering h\_ <sup>d</sup> is appropriately bounded.

It is important to point out that the orientation of a DDMR is always tangent to the path being followed. Moreover, the desired trajectory defines the desired linear speed ud, which means that the robot will be moving either forward or backwards. Therefore, it is not necessary for the controller to explicitly control the robot's orientation to make it successfully follow a trajectory with a desired orientation.

For the stability analysis of the kinematic controller, it is supposed a perfect velocity tracking, which allows equating (19) and (20) under the assumption of u � ud and ω � ωd, which means that the dynamic effects are, at this moment, ignored. Then, the closed-loop equation is obtained in terms of the velocity errors, which is

$$
\begin{bmatrix} \dot{\tilde{\boldsymbol{x}}} \\ \dot{\tilde{\boldsymbol{y}}} \end{bmatrix} + \begin{bmatrix} l\_{\boldsymbol{x}} & \mathbf{0} \\ \mathbf{0} & l\_{\boldsymbol{y}} \end{bmatrix} \begin{bmatrix} \tanh\left(\frac{k\_{\boldsymbol{x}}}{l\_{\boldsymbol{x}}} \tilde{\boldsymbol{x}}\right) \\\\ \tanh\left(\frac{k\_{\boldsymbol{y}}}{l\_{\boldsymbol{y}}} \tilde{\boldsymbol{y}}\right) \end{bmatrix} = \begin{bmatrix} \mathbf{0} \\ \mathbf{0} \end{bmatrix}. \tag{21}
$$

Now, the output error vector h~ (21) can be written as

$$\dot{\tilde{\mathbf{h}}} = -\left[l\_x \tanh\left(\frac{k\_x}{l\_x}\tilde{\mathbf{x}}\right) \quad l\_y \tanh\left(\frac{k\_y}{l\_y}\tilde{\mathbf{y}}\right)\right]^T,\tag{22}$$

which has an unique equilibrium point at the origin. To conclude the stability analysis of such equilibrium, <sup>V</sup> <sup>¼</sup> <sup>1</sup> <sup>2</sup> <sup>h</sup>~<sup>T</sup>h<sup>~</sup> <sup>&</sup>gt; 0 is considered as the Lyapunov's candidate function. Its first time derivative is

$$\dot{V} = \tilde{\mathbf{h}}^T \dot{\tilde{\mathbf{h}}} = -\tilde{\mathbf{x}} l\_x \tanh\left(\frac{k\_x}{l\_x} \tilde{\mathbf{x}}\right) - \tilde{y} l\_y \tanh\left(\frac{k\_y}{l\_y} \tilde{y}\right) < 0, \forall \tilde{\mathbf{h}}.\tag{23}$$

Regarding these results, one can immediately conclude that the system characterized so far has a globally asymptotically stable equilibrium at the origin, which means that the position errors ~x tðÞ! 0 and ~y tðÞ! 0 as t ! ∞. This result will be revisited latter, after adding a dynamic controller to the system in order to implement the whole control scheme.

Remark. Considering the case in which the reference is a fixed destination point, instead of a trajectory, the robot reaches such a point and stops there. Assuming u � ud and ω � ωd, Eq. (20) guarantees that ω ¼ 0 when ~x ¼ 0 and ~y ¼ 0, therefore ψðÞ!t ψconstant.

#### 4.2. Dynamic compensation controller

The inverse kinematics described by Eq. (19) is valid only for a 6¼ 0. This is the reason why we prefer to adopt this model instead of the classical unicycle model, as discussed earlier. Consid-

ky > 0 are the controller gains; lx, ly ∈ R are saturation constants; and a > 0. The tanh terms are included to limit the values of the desired velocities vd to avoid saturation of the robot

It is important to point out that the orientation of a DDMR is always tangent to the path being followed. Moreover, the desired trajectory defines the desired linear speed ud, which means that the robot will be moving either forward or backwards. Therefore, it is not necessary for the controller to explicitly control the robot's orientation to make it successfully follow a trajectory

For the stability analysis of the kinematic controller, it is supposed a perfect velocity tracking, which allows equating (19) and (20) under the assumption of u � ud and ω � ωd, which means that the dynamic effects are, at this moment, ignored. Then, the closed-loop equation is

3 5

y\_

<sup>x</sup>\_ <sup>d</sup> <sup>þ</sup> lx tanh kx

<sup>d</sup> <sup>þ</sup> ly tanh ky

<sup>T</sup> is the vector of desired velocities given by the kinematic controller;

� �<sup>T</sup> are the vectors of actual and desired coordinates of the point of

lx ~x � �

, (20)

<sup>d</sup> is appropriately bounded.

: (21)

, (22)

< 0, ∀h~: (23)

ly ~y � �

<sup>T</sup> is the vector of position errors given by hd � <sup>h</sup>; kx <sup>&</sup>gt; 0 and

cosψ sinψ

sin<sup>ψ</sup> <sup>1</sup> a cosψ

ering (19), the adopted control law is

<sup>T</sup> and hd <sup>¼</sup> xd yd

interest <sup>h</sup>, respectively; <sup>h</sup><sup>~</sup> <sup>¼</sup> ½ � <sup>~</sup><sup>x</sup> <sup>~</sup><sup>y</sup>

for which vd ¼ ½ � ud ω<sup>d</sup>

32 Applications of Mobile Robots

with a desired orientation.

equilibrium, <sup>V</sup> <sup>¼</sup> <sup>1</sup>

derivative is

h ¼ ½ � x y

ud ωd � �

obtained in terms of the velocity errors, which is

~\_ x ~\_ y " #

Now, the output error vector h~ (21) can be written as

<sup>V</sup>\_ <sup>¼</sup> <sup>h</sup>~<sup>T</sup> ~\_

~\_

þ

lx 0 0 ly

<sup>h</sup> ¼ � lx tanh kx

<sup>h</sup> ¼ �~xlx tanh kx

� � tanh kx

lx ~x � �

lx ~x � �

which has an unique equilibrium point at the origin. To conclude the stability analysis of such

Regarding these results, one can immediately conclude that the system characterized so far has a globally asymptotically stable equilibrium at the origin, which means that the position errors

lx ~x � �

ly tanh ky ly ~y

<sup>2</sup> <sup>h</sup>~<sup>T</sup>h<sup>~</sup> <sup>&</sup>gt; 0 is considered as the Lyapunov's candidate function. Its first time

� <sup>~</sup>yly tanh ky

ly ~y � �

tanh ky ly ~y � �

h i � � <sup>T</sup>

¼

2 4

� 1 a

actuators in case the position errors h~ are too big, considering h\_

Now, the use of the proposed dynamic model and its properties is illustrated via the design of a dynamic compensation controller. It receives the desired velocities vd from the kinematic controller and generates a pair of linear and angular velocity references vr for the robot servos, as shown in Figure 2. First, let us define the vector of modified velocities v<sup>0</sup> <sup>d</sup> as

$$\mathbf{v}'\_{\mathbf{d}} = \begin{bmatrix} u'\_d \\ \omega\_d \end{bmatrix} = \begin{bmatrix} i & 0 \\ 0 & 1 \end{bmatrix} \begin{bmatrix} \omega\_d \\ \omega\_d \end{bmatrix} \tag{24}$$

and the vector of velocity errors is given by v~<sup>0</sup> ¼v<sup>0</sup> <sup>d</sup> � v<sup>0</sup> .

Regarding parametric uncertainties, the proposed dynamic compensation control law is

$$\mathbf{v}\_{\mathbf{r}} = \widehat{\mathbf{H}} \left( \dot{\mathbf{v}}\_{\mathbf{d}}^{\prime} + \mathbf{T}(\ddot{\mathbf{v}}^{\prime}) \right) + \widehat{\mathbf{C}} \mathbf{v}\_{\mathbf{d}}^{\prime} + \widehat{\mathbf{F}} \mathbf{v}\_{\mathbf{d}^{\prime}}^{\prime} \tag{25}$$

where Hb , Cb, and bF are estimates of H, C, and F, respectively, T v~<sup>0</sup> ð Þ¼ lu 0 0 l<sup>ω</sup> � � tanh ku lu iu~ � � tanh <sup>k</sup><sup>ω</sup> lω ω~ � � 2 6 6 6 4 3 7 7 7 5 , ku > 0 and k<sup>ω</sup> > 0 are gain constants, lu ∈ R and l<sup>ω</sup> ∈ R are satura-

tion constants, and ω~ ¼ ω<sup>d</sup> � ω and u~ ¼ ud � u are the current velocity errors. The term T v~<sup>0</sup> ð Þ provides a saturation in order to guarantee that the commands to be sent to the robot are always below the corresponding physical limits, considering that v<sup>0</sup> <sup>d</sup> and v\_<sup>0</sup> <sup>d</sup> are bounded to appropriate values.

In this chapter, we consider that the dynamic parameters are exactly known, that is, θb ¼ θ. This means that Hb ¼ H, Cb ¼ C, and bF ¼ F. The analysis considering parametric error is presented in [7, 21].

Using the Lyapunov candidate function <sup>V</sup> <sup>¼</sup> <sup>1</sup> <sup>2</sup> <sup>v</sup>~0THv~<sup>0</sup> <sup>&</sup>gt; 0, and considering that the dynamic parameters are constant, one has

$$
\dot{V} = -\tilde{\mathbf{v}}'^{\mathsf{T}} \mathbf{H} \mathbf{T}(\tilde{\mathbf{v}}') - \tilde{\mathbf{v}}'^{\mathsf{T}} \mathbf{C} \tilde{\mathbf{v}}' - \tilde{\mathbf{v}}'^{\mathsf{T}} \mathbf{F} \tilde{\mathbf{v}}'.\tag{26}
$$

Observing property 5, of antisymmetry of C, the derivative of the Lyapunov function can be written as

$$
\dot{V} = -\tilde{\mathbf{v}}'^{\mathsf{T}} \mathbf{H} \mathbf{T}(\tilde{\mathbf{v}}') - \tilde{\mathbf{v}}'^{\mathsf{T}} \mathbf{F} \tilde{\mathbf{v}}'.\tag{27}
$$

According to Property 1, H is symmetric and positive definite. The terms of T v~<sup>0</sup> ð Þ have the same sign of the terms of v~<sup>0</sup> . Property 3 states that F is symmetric and positive definite if θ<sup>6</sup> > �ð Þ θ5=I � θ<sup>3</sup> Iu, condition that was shown to hold for our robot. Therefore, one can conclude that <sup>V</sup>\_ <sup>&</sup>lt; 0, that is, <sup>v</sup>~<sup>0</sup> <sup>∈</sup> <sup>L</sup><sup>∞</sup> and <sup>v</sup>~<sup>0</sup> ! <sup>0</sup> with <sup>t</sup> ! <sup>∞</sup>, and <sup>v</sup><sup>~</sup> <sup>∈</sup>L<sup>∞</sup> and <sup>v</sup><sup>~</sup> ! <sup>0</sup> with <sup>t</sup> ! <sup>∞</sup>.

Regarding the kinematic controller, it has been shown [7] that a sufficient condition for the asymptotic stability is

$$\|\|\tilde{\mathbf{h}}\|\| > \frac{\|\|\mathbf{A}\tilde{\mathbf{v}}\|\|}{\min} (k\_x, k\_y),\tag{28}$$

Figure 4 illustrates the results of 2 experiments, both without load. Figure 4(a) shows the 8 shape path followed by the robot without load when controlled by KC and DC. Robot path was recovered through its odometry. One can notice that the path followed by the robot is slightly different under KC or DC. The robot's linear and angular velocities also change along

Figure 3. 8-shape reference path to be followed by the robot. Initial reference position is 0ð Þ :0; 0:0 m and the direction of

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

http://dx.doi.org/10.5772/intechopen.79397

35

motion is indicated in the figure. The robot starts at position 0ð Þ :0; 0:0 m with orientation 0�.

A better visualization of the tracking error is given by Figure 5, which shows the evolution of the distance error during the experiments without load. The distance error is defined as the

Figure 4. Experiments without load: (a) robot path; (b) linear and angular velocities. In all graphs, the black line represents the results for the case in which the dynamic compensation (DC) is active, while the red line represents the

the path, as shown in Figure 4(b).

results for the kinematic controller (KC).

where <sup>A</sup> <sup>¼</sup> cos<sup>ψ</sup> �<sup>a</sup> sin <sup>ψ</sup> sinψ a cos ψ : Because <sup>v</sup>~ðÞ!<sup>t</sup> <sup>0</sup>, the condition (28) is asymptotically verified

for any value of <sup>h</sup>~. Consequently, the tracking control error <sup>h</sup>~ðÞ!<sup>t</sup> <sup>0</sup>, thus accomplishing the control objective.

To sum up, by using a control structure as shown in Figure 2 with a dynamic compensation controller given by Eq. (25), different motion controllers can be applied. In our example, the trajectory tracking controller given by Eq. (20) was used. This is the system that we have implemented and for which we present some experimental results in Section 5.

#### 5. Experimental results

In this section, we present some experimental results using a Pioneer 3-DX, from Adept Mobile Robots. In all experiments, the robot starts at position 0ð Þ :0; 0:0 m with orientation 0�, and should follow an 8-shape trajectory also starting at 0ð Þ :0; 0:0 m. The trajectory to be followed by the robot is represented by a sequence of desired positions hd and velocities h\_ <sup>d</sup>, both varying in time. The reference path is illustrated in Figure 3.

We have implemented the control structure shown on Figure 2 using the control laws given by Eqs. (20) and (25). In total, we have executed 10 experiments for each controller, from now on referred to as KC (kinematics controller) and DC (dynamic compensation). In the case of KC, the robot receives as commands the values vd calculated by the kinematics controller and there is no dynamic compensation. On the other hand, in the case of DC, the dynamic compensation controller is active and the robot receives as commands the values of vr calculated by the dynamic compensation controller. We have repeated the experiments for four cases: KC with load, KC without load, DC with load, and DC without load. The load consists of a weight of 24:8 kg placed on top of the robot, while the original weight of the robot is 10:4 kg.

The following parameters were used in all experiments: a ¼ 0:15 m, sample time of 0.1 s (this is the sample time of the Pioneer 3-DX); controller gains kx ¼ 0:1, ky ¼ 0:1, ku ¼ 4, kw ¼ 4, and saturation constants lx ¼ 0:1, ly ¼ 0:1, lu ¼ 1, lw ¼ 1. The robot used in the experiments is a Pioneer 3-DX without LASER scanner, therefore the parameters used in the dynamic compensation controller are the ones in column P3 from Table 1.

According to Property 1, H is symmetric and positive definite. The terms of T v~<sup>0</sup> ð Þ have the

θ<sup>6</sup> > �ð Þ θ5=I � θ<sup>3</sup> Iu, condition that was shown to hold for our robot. Therefore, one can conclude that <sup>V</sup>\_ <sup>&</sup>lt; 0, that is, <sup>v</sup>~<sup>0</sup> <sup>∈</sup> <sup>L</sup><sup>∞</sup> and <sup>v</sup>~<sup>0</sup> ! <sup>0</sup> with <sup>t</sup> ! <sup>∞</sup>, and <sup>v</sup><sup>~</sup> <sup>∈</sup>L<sup>∞</sup> and <sup>v</sup><sup>~</sup> ! <sup>0</sup> with <sup>t</sup> ! <sup>∞</sup>. Regarding the kinematic controller, it has been shown [7] that a sufficient condition for the

min kx; ky

for any value of <sup>h</sup>~. Consequently, the tracking control error <sup>h</sup>~ðÞ!<sup>t</sup> <sup>0</sup>, thus accomplishing the

To sum up, by using a control structure as shown in Figure 2 with a dynamic compensation controller given by Eq. (25), different motion controllers can be applied. In our example, the trajectory tracking controller given by Eq. (20) was used. This is the system that we have

In this section, we present some experimental results using a Pioneer 3-DX, from Adept Mobile Robots. In all experiments, the robot starts at position 0ð Þ :0; 0:0 m with orientation 0�, and should follow an 8-shape trajectory also starting at 0ð Þ :0; 0:0 m. The trajectory to be followed by the robot is represented by a sequence of desired positions hd and velocities h\_

We have implemented the control structure shown on Figure 2 using the control laws given by Eqs. (20) and (25). In total, we have executed 10 experiments for each controller, from now on referred to as KC (kinematics controller) and DC (dynamic compensation). In the case of KC, the robot receives as commands the values vd calculated by the kinematics controller and there is no dynamic compensation. On the other hand, in the case of DC, the dynamic compensation controller is active and the robot receives as commands the values of vr calculated by the dynamic compensation controller. We have repeated the experiments for four cases: KC with load, KC without load, DC with load, and DC without load. The load consists of a weight of

The following parameters were used in all experiments: a ¼ 0:15 m, sample time of 0.1 s (this is the sample time of the Pioneer 3-DX); controller gains kx ¼ 0:1, ky ¼ 0:1, ku ¼ 4, kw ¼ 4, and saturation constants lx ¼ 0:1, ly ¼ 0:1, lu ¼ 1, lw ¼ 1. The robot used in the experiments is a Pioneer 3-DX without LASER scanner, therefore the parameters used in the dynamic compen-

24:8 kg placed on top of the robot, while the original weight of the robot is 10:4 kg.

implemented and for which we present some experimental results in Section 5.

varying in time. The reference path is illustrated in Figure 3.

sation controller are the ones in column P3 from Table 1.

<sup>∥</sup>h~<sup>∥</sup> <sup>&</sup>gt; <sup>∥</sup>Av~<sup>∥</sup>

. Property 3 states that F is symmetric and positive definite if

: Because v~ðÞ!t 0, the condition (28) is asymptotically verified

, (28)

<sup>d</sup>, both

same sign of the terms of v~<sup>0</sup>

where <sup>A</sup> <sup>¼</sup> cos<sup>ψ</sup> �<sup>a</sup> sin <sup>ψ</sup>

5. Experimental results

sinψ a cos ψ 

asymptotic stability is

34 Applications of Mobile Robots

control objective.

Figure 3. 8-shape reference path to be followed by the robot. Initial reference position is 0ð Þ :0; 0:0 m and the direction of motion is indicated in the figure. The robot starts at position 0ð Þ :0; 0:0 m with orientation 0�.

Figure 4 illustrates the results of 2 experiments, both without load. Figure 4(a) shows the 8 shape path followed by the robot without load when controlled by KC and DC. Robot path was recovered through its odometry. One can notice that the path followed by the robot is slightly different under KC or DC. The robot's linear and angular velocities also change along the path, as shown in Figure 4(b).

A better visualization of the tracking error is given by Figure 5, which shows the evolution of the distance error during the experiments without load. The distance error is defined as the

Figure 4. Experiments without load: (a) robot path; (b) linear and angular velocities. In all graphs, the black line represents the results for the case in which the dynamic compensation (DC) is active, while the red line represents the results for the kinematic controller (KC).

Figure 5. Evolution of tracking error without load. The black line represents the error for the case in which the dynamic compensation (DC) is active, while the red line represents the error for the kinematic controller (KC). The corresponding values of IAE<sup>30</sup> for this experiment are also shown in the figure.

instantaneous distance between the desired position hd and the actual robot position h. It can be noticed that the distance error is similar for KC and DC in the first part of the path. At the beginning of the experiment, the tracking error increases quite a lot, reaching almost 1:0 m. This happens because the robot needs to accelerate from zero to catch up with the reference trajectory. After a few seconds, the error starts to decrease and around 25 30 s, the robot follows the trajectory at normal speed. From this point on, it is clear that the average error is smaller when the DC is active.

Figure 6(a) shows the 8-shape path followed by the robot when carrying the load and controlled by KC and DC. One can notice that the path followed by the robot is slightly different under KC or DC, and there is more distortion in the path when compared to the case in which the robot carries no load. The robot's linear and angular velocities also change along the path, as shown in Figure 6(b), and are very similar to the previous case.

The tracking error is given by Figure 7, which shows the evolution of the distance error during the experiments with load. As before, the robot needs to accelerate from zero to catch up with the reference trajectory, which causes the tracking error to increase in the first part of the experiments. But, in this case, the error in the first part of the experiment is actually higher for DC. This happens because the dynamic parameters used in the dynamic compensation controller remained unchanged during all experiments, with and without load. This means that the case in which the robot is carrying load is unfavorable for the dynamic compensation controller because the dynamics is not properly compensated, causing the error to increase. Even so, after about 30 s, the tracking error of DC gets smaller than the error for KC.

To evaluate the performance of the system we have calculated the IAE performance index,

Figure 7. Evolution of tracking error with load. The black line represents the error for the case in which the dynamic compensation (DC) is active, while the red line represents the error for the kinematic controller (KC). The corresponding

Figure 6. Experiments with load: (a) robot path; (b) linear and angular velocities. In all graphs the black line represents the results for the case in which the dynamic compensation (DC) is active, while the red line represents the results for the

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

http://dx.doi.org/10.5772/intechopen.79397

37

period of integration. The average and standard deviation values of IAE for all experiments are reported in Table 2. There, IAEtot was calculated considering t<sup>2</sup> ¼ 75 s and t<sup>1</sup> ¼ 0, that is, for the total period of each experiment. By its turn, the value of IAE<sup>30</sup> was calculated only for the

<sup>x</sup>~<sup>2</sup> <sup>þ</sup> <sup>y</sup>~<sup>2</sup> <sup>p</sup> is the instantaneous distance error and <sup>t</sup><sup>2</sup> � <sup>t</sup><sup>1</sup> is the

where IAE <sup>¼</sup> <sup>Ð</sup><sup>t</sup><sup>2</sup>

kinematic controller (KC).

<sup>t</sup><sup>1</sup> E tð Þdt, E tðÞ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

values of IAE<sup>30</sup> for this experiment are also shown in the figure.

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots http://dx.doi.org/10.5772/intechopen.79397 37

Figure 6. Experiments with load: (a) robot path; (b) linear and angular velocities. In all graphs the black line represents the results for the case in which the dynamic compensation (DC) is active, while the red line represents the results for the kinematic controller (KC).

instantaneous distance between the desired position hd and the actual robot position h. It can be noticed that the distance error is similar for KC and DC in the first part of the path. At the beginning of the experiment, the tracking error increases quite a lot, reaching almost 1:0 m. This happens because the robot needs to accelerate from zero to catch up with the reference trajectory. After a few seconds, the error starts to decrease and around 25 30 s, the robot follows the trajectory at normal speed. From this point on, it is clear that the average error is

Figure 5. Evolution of tracking error without load. The black line represents the error for the case in which the dynamic compensation (DC) is active, while the red line represents the error for the kinematic controller (KC). The corresponding

Figure 6(a) shows the 8-shape path followed by the robot when carrying the load and controlled by KC and DC. One can notice that the path followed by the robot is slightly different under KC or DC, and there is more distortion in the path when compared to the case in which the robot carries no load. The robot's linear and angular velocities also change along the path,

The tracking error is given by Figure 7, which shows the evolution of the distance error during the experiments with load. As before, the robot needs to accelerate from zero to catch up with the reference trajectory, which causes the tracking error to increase in the first part of the experiments. But, in this case, the error in the first part of the experiment is actually higher for DC. This happens because the dynamic parameters used in the dynamic compensation controller remained unchanged during all experiments, with and without load. This means that the case in which the robot is carrying load is unfavorable for the dynamic compensation controller because the dynamics is not properly compensated, causing the error to increase. Even so, after

as shown in Figure 6(b), and are very similar to the previous case.

about 30 s, the tracking error of DC gets smaller than the error for KC.

smaller when the DC is active.

36 Applications of Mobile Robots

values of IAE<sup>30</sup> for this experiment are also shown in the figure.

Figure 7. Evolution of tracking error with load. The black line represents the error for the case in which the dynamic compensation (DC) is active, while the red line represents the error for the kinematic controller (KC). The corresponding values of IAE<sup>30</sup> for this experiment are also shown in the figure.

To evaluate the performance of the system we have calculated the IAE performance index, where IAE <sup>¼</sup> <sup>Ð</sup><sup>t</sup><sup>2</sup> <sup>t</sup><sup>1</sup> E tð Þdt, E tðÞ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi <sup>x</sup>~<sup>2</sup> <sup>þ</sup> <sup>y</sup>~<sup>2</sup> <sup>p</sup> is the instantaneous distance error and <sup>t</sup><sup>2</sup> � <sup>t</sup><sup>1</sup> is the period of integration. The average and standard deviation values of IAE for all experiments are reported in Table 2. There, IAEtot was calculated considering t<sup>2</sup> ¼ 75 s and t<sup>1</sup> ¼ 0, that is, for the total period of each experiment. By its turn, the value of IAE<sup>30</sup> was calculated only for the


and modeling the dynamics or the robot's actuators. We have shown that such model and its properties are useful on the design and stability analysis of a dynamic compensation controller for a differential-drive mobile robot. Moreover, because the mathematical structure of (12) is similar to the classical torque-based model, classical strategies for controller design [8, 26] can be adapted for designing controllers for mobile robots using the model presented in this chapter.

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

http://dx.doi.org/10.5772/intechopen.79397

39

The dynamic model presented in this chapter can be used in connection with other kinematic controllers designed for commercial mobile robots, even in the context of coordinated control of multi-robot formations [27]. This integration requires no change on the original controller equations since the dynamic model accepts the same velocity commands as commercial robots. We invite the interested reader to download our toolbox for MATLAB/Simulink®, which include blocks to simulate the differential-drive kinematics and dynamics, a kinematic controller and two dynamic compensation controllers, one of which being the one presented in this chapter [28].

The authors thank the Institute of Engineering, Hanze University of Applied Sciences, for the

\* and Alexandre Santos Brandão2

1 Institute of Engineering, Hanze University of Applied Sciences, Assen, The Netherlands

[1] Siegwart R, Nourbakhsh IR, Scaramuzza D. Introduction to Autonomous Mobile Robots.

[2] Birk A, Kenn H. RoboGuard, a teleoperated mobile security robot. Control Engineering

[3] Prassler E et al. A short history of cleaning robots. Autonomous Robots. 2000;9(3):211-226 [4] Stouten B, de Graaf AJ. Cooperative transportation of a large object-development of an industrial application. In: IEEE International Conference on Robotics and Automation. Vol. 3. 2004. pp. 2450-2455. New Orleans, LA, USA: IEEE; DOI: 10.1109/ROBOT.2004.1307428 [5] Andaluz VH et al. Robust control with dynamic compensation for human-wheelchair system. In: Intelligent Robotics and Applications. Switzerland: Springer; 2014. pp. 376-389

2 Department of Electrical Engineering, Federal University of Viçosa, Viçosa, Brazil

partial financial support given for the publication of this chapter.

\*Address all correspondence to: fe.nascimento.martins@pl.hanze.nl

2nd ed. London, England: MIT Press; 2011

Practice. 2002;10:1259-1264

Acknowledgements

Author details

References

Felipe Nascimento Martins<sup>1</sup>

Here, IAE <sup>¼</sup> <sup>Ð</sup><sup>t</sup><sup>2</sup> <sup>t</sup><sup>1</sup> <sup>∣</sup>E tð Þ∣dt, E tðÞ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi <sup>x</sup>~<sup>2</sup> <sup>þ</sup> <sup>y</sup>~<sup>2</sup> <sup>p</sup> is the instantaneous distance error, and <sup>t</sup><sup>2</sup> � <sup>t</sup><sup>1</sup> is the period of integration. For IAEtot, t<sup>2</sup> ¼ 75 s and t<sup>1</sup> ¼ 0. For IAE30, t<sup>2</sup> ¼ 75 s and t<sup>1</sup> ¼ 35 s.

Table 2. Average and standard deviation of IAE performance index calculated for experiments with and without load (lower value is better, highlighted in bold).

final 30 seconds of each experiment, that is, considering t<sup>2</sup> ¼ 75s and t<sup>1</sup> ¼ 35s. Therefore, IAE<sup>30</sup> gives a good indication of the performance of the system after the error due to initial acceleration have faded out. From the results highlighted in bold in Table 2, it is clear that the performance of the system with the dynamic compensation controller is better in the long run because the correspondent values of IAE<sup>30</sup> are about 50% of those for the kinematic controller. This is true even for the case in which the robot is carrying load.

It is important to emphasize that the dynamic parameters used in the dynamic compensation controller remained unchanged during all experiments, which means that the dynamics is not properly compensated when carrying load. This is illustrated by the fact that IAEtot is bigger when the dynamic compensation is active and the robot is carrying load. Even so, in our experiments the performance was better in the long run when the dynamic compensation controller remained active.

One should notice that an increase in controller gains kx and ky could result in better performance (smaller tracking error), especially when the robot is carrying load. Nevertheless, we kept the same values of controller gains during all experiments to be able to compare the results.

#### 6. Conclusion

In this chapter, we illustrate that the performance (in term of IAE) of a motion control system for a mobile robot can be up to 50% better under certain conditions when dynamic compensation is included. Such dynamic compensation can be implemented as shown in Figure 2, in which Eq. (25) is used with parameters identified via the procedure described in Section 3.2.

It is worth mentioning that the values of controller gains used in the experiments here reported were not optimum. The values of the gains were chosen empirically so that we could compare different cases. Optimization of controller gains can be executed to reduce tracking error, energy consumption, or a weighted combination of both, as shown in [25]. This means that the performance of the overall system could potentially be better than reported here.

We also presented a formulation of a dynamic model for differential-drive mobile robots, and discussed its mathematical properties. When compared to the classical dynamic model based on torques, the model used in this chapter has the advantages of accepting velocities as inputs, and modeling the dynamics or the robot's actuators. We have shown that such model and its properties are useful on the design and stability analysis of a dynamic compensation controller for a differential-drive mobile robot. Moreover, because the mathematical structure of (12) is similar to the classical torque-based model, classical strategies for controller design [8, 26] can be adapted for designing controllers for mobile robots using the model presented in this chapter.

The dynamic model presented in this chapter can be used in connection with other kinematic controllers designed for commercial mobile robots, even in the context of coordinated control of multi-robot formations [27]. This integration requires no change on the original controller equations since the dynamic model accepts the same velocity commands as commercial robots. We invite the interested reader to download our toolbox for MATLAB/Simulink®, which include blocks to simulate the differential-drive kinematics and dynamics, a kinematic controller and two dynamic compensation controllers, one of which being the one presented in this chapter [28].

#### Acknowledgements

final 30 seconds of each experiment, that is, considering t<sup>2</sup> ¼ 75s and t<sup>1</sup> ¼ 35s. Therefore, IAE<sup>30</sup> gives a good indication of the performance of the system after the error due to initial acceleration have faded out. From the results highlighted in bold in Table 2, it is clear that the performance of the system with the dynamic compensation controller is better in the long run because the correspondent values of IAE<sup>30</sup> are about 50% of those for the kinematic controller.

Table 2. Average and standard deviation of IAE performance index calculated for experiments with and without load

With load Without load

Kinematic controller 14:30 � 0:66 3:10 � 0:05 14:08 � 0:18 3:11 � 0:12 Dynamic compensation 15:39 � 0:42 1:41 � 0:13 13:05 � 0:07 1:29 � 0:02

IAEtot IAE<sup>30</sup> IAEtot IAE<sup>30</sup>

<sup>x</sup>~<sup>2</sup> <sup>þ</sup> <sup>y</sup>~<sup>2</sup> <sup>p</sup> is the instantaneous distance error, and <sup>t</sup><sup>2</sup> � <sup>t</sup><sup>1</sup> is the period of integration. For

It is important to emphasize that the dynamic parameters used in the dynamic compensation controller remained unchanged during all experiments, which means that the dynamics is not properly compensated when carrying load. This is illustrated by the fact that IAEtot is bigger when the dynamic compensation is active and the robot is carrying load. Even so, in our experiments the performance was better in the long run when the dynamic compensation

One should notice that an increase in controller gains kx and ky could result in better performance (smaller tracking error), especially when the robot is carrying load. Nevertheless, we kept the

In this chapter, we illustrate that the performance (in term of IAE) of a motion control system for a mobile robot can be up to 50% better under certain conditions when dynamic compensation is included. Such dynamic compensation can be implemented as shown in Figure 2, in which Eq. (25) is used with parameters identified via the procedure described in Section 3.2. It is worth mentioning that the values of controller gains used in the experiments here reported were not optimum. The values of the gains were chosen empirically so that we could compare different cases. Optimization of controller gains can be executed to reduce tracking error, energy consumption, or a weighted combination of both, as shown in [25]. This means that

same values of controller gains during all experiments to be able to compare the results.

the performance of the overall system could potentially be better than reported here.

We also presented a formulation of a dynamic model for differential-drive mobile robots, and discussed its mathematical properties. When compared to the classical dynamic model based on torques, the model used in this chapter has the advantages of accepting velocities as inputs,

This is true even for the case in which the robot is carrying load.

controller remained active.

6. Conclusion

Here, IAE <sup>¼</sup> <sup>Ð</sup><sup>t</sup><sup>2</sup>

38 Applications of Mobile Robots

<sup>t</sup><sup>1</sup> <sup>∣</sup>E tð Þ∣dt, E tðÞ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

(lower value is better, highlighted in bold).

IAEtot, t<sup>2</sup> ¼ 75 s and t<sup>1</sup> ¼ 0. For IAE30, t<sup>2</sup> ¼ 75 s and t<sup>1</sup> ¼ 35 s.

The authors thank the Institute of Engineering, Hanze University of Applied Sciences, for the partial financial support given for the publication of this chapter.

#### Author details

Felipe Nascimento Martins<sup>1</sup> \* and Alexandre Santos Brandão2

\*Address all correspondence to: fe.nascimento.martins@pl.hanze.nl


#### References


[6] Morin P, Samson C. Motion control of wheeled mobile robots. In: Springer Handbook of Robotics. Heidelberg, Germany: Springer; 2008. pp. 799-826

[21] Martins FN, Sarcinelli-Filho M, Carelli R. A velocity-based dynamic model and its properties for differential drive mobile robots. Journal of Intelligent & Robotic Systems. 2017;

Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots

http://dx.doi.org/10.5772/intechopen.79397

41

[22] Fukao T, Nakagawa H, Adachi N. Adaptive tracking control of a Nonholonomic mobile

[23] Kim MS, Shin JH, Lee JJ. Design of a robust adaptive controller for a mobile robot. In: Proc. of the IEEE/RSJ Int. Conf. On Intelligent Robots and Systems. Vol. 3. Takamatsu, Japan:

[24] Laut J. A dynamic parameter identification method for migrating control strategies between heterogeneous wheeled mobile robots. [PhD thesis]. Worcester Polytechnic Insti-

[25] Martins FN, Almeida GM. Tuning a velocity-based dynamic controller for unicycle mobile robots with genetic algorithm. In: Joranadas Argentinas de Robótica. Olavarría, Argentina:

[26] Spong MW, Hutchinson S, Vidyasagar M. Robot Modeling and Control. Vol. 3. New York:

[27] Brandao AS et al. A multi-layer control scheme for multi-robot formations with adaptive dynamic compensation. In: IEEE International Conference on Mechatronics, 2009. ICM

[28] F.N. Martins. Velocity-based Dynamic Model and Adaptive Controller for Differential Steered Mobile Robot. 2017. Available from: http://www.mathworks.com/matlabcentral/

robot. IEEE Transactions on Robotics and Automation. 2000;16(5):609-615

Universidad Nacional del Centro de la Provincia de Buenos Aires; 2012

IEEE; 2000. pp. 1816-1821. DOI: 10.1109/IROS.2000.895235

85(2):277-292

tute; 2011

Wiley; 2006

fileexchange/44850

2009. Malaga, Spain: IEEE; 2009


[21] Martins FN, Sarcinelli-Filho M, Carelli R. A velocity-based dynamic model and its properties for differential drive mobile robots. Journal of Intelligent & Robotic Systems. 2017; 85(2):277-292

[6] Morin P, Samson C. Motion control of wheeled mobile robots. In: Springer Handbook of

[7] Martins FN et al. An adaptive dynamic controller for autonomous mobile robot trajectory tracking. Control Engineering Practice. 2008;16:1354-1363. DOI: 10.1016/j.conengprac.2008.

[8] Fierro R, Lewis FL. Control of a nonholonomic mobile robot: Backstepping kinematics

[9] Das T, Kar IN. Design and implementation of an adaptive fuzzy logic-based controller for wheeled mobile robots. IEEE Transactions on Control Systems Technology. 2006;14(3):501-510

[10] Lapierre L, Zapata R, Lepinay P. Combined path-following and obstacle avoidance control of a wheeled robot. The Int. Journal of Robotics Research. 2007;26(4):361-375

[11] Shojaei K, Mohammad Shahri A, Tarakameh A. Adaptive feedback linearizing control of nonholonomic wheeled mobile robots in presence of parametric and nonparametric uncertainties. Robotics and Computer-Integrated Manufacturing. 2011;27(1):194-204

[12] Wang K. Near-optimal tracking control of a Nonholonomic mobile robot with uncer-

[13] Do KD. Bounded controllers for global path tracking control of unicycle-type mobile robots. Robotics and Autonomous Systems. 2013;61(8):775-784. ISSN: 0921-8890. https://

[14] Onat A, Ozkan M. Dynamic adaptive trajectory tracking control of nonholonomic mobile robots using multiple models approach. Advanced Robotics. 2015;29(14):913-928

[15] Antonini P, Ippoliti G, Longhi S. Learning control of mobile robots using a multiprocessor

[16] De La Cruz C, Carelli R. Dynamic model based formation control and obstacle avoidance

[17] Chen CY et al. Design and implementation of an adaptive sliding-mode dynamic controller for wheeled mobile robots. Mechatronics. 2009;19(2):156-166. ISSN: 0957-4158

[18] De La Cruz C, Celeste WC, Bastos-Filho TF. A robust navigation system for robotic

[19] Rossomando FG, Soria C, Carelli R. Adaptive neural sliding mode compensator for a class of nonlinear systems with unmodeled uncertainties. Engineering Applications of Artificial

[20] Utstumo T, Berge TW, Gravdahl JT. Non-linear model predictive control for constrained robot navigation in row crops. In: IEEE International Conference on Industrial Technology

(ICIT 2015). Seville, Spain: IEEE; 2015. DOI: 10.1109/ICIT.2015.7125124

tainties. International Journal of Advanced Robotic Systems. 2012;9(66):1-11

doi.org/10.1016/j.robot.2013.04.014

Intelligence. 2013;26(10):2251-2259

system. Control Engineering Practice. 2006;14:1279-1295

of multi-robot systems. Robotica. 2008;26(03):345-356

wheelchairs. Control Engineering Practice. 2011;19(6):575-590

Robotics. Heidelberg, Germany: Springer; 2008. pp. 799-826

into dynamics. Journal of Robotic Systems. 1997;14(3):149-163

03.004

40 Applications of Mobile Robots


**Chapter 3**

Provisional chapter

**Theoretical and Experimental Collaborative Area**

DOI: 10.5772/intechopen.78940

This chapter is concerned with the development of collaborative control schemes for mobile ground robots for area coverage purposes. The simplest scheme assumes point omnidirectional robots with heterogeneous circular sensing patterns. Using information from their spatial neighbors, each robot (agent) computes its cell relying on the power diagram partitioning. If there is uncertainty in inferring the locations of these robots, the Additively Weighted Guaranteed Voronoi scheme is employed resulting in a rather conservative performance. The aforementioned schemes are enhanced by using a Voronoifree coverage scheme that relies on the knowledge of any arbitrary sensing pattern employed by the agents. Experimental results are offered to highlight the efficiency of the

Keywords: area coverage, multiagent systems, mobile robot systems, distributed control,

The problem of area coverage is one that has been widely studied in the past decade and consists of the deployment of a sensor-equipped mobile robot team. It is usually categorized as either blanket or sweep coverage. In blanket or static coverage the goal of the robot team is a final static configuration at which an objective function is maximized [1–3]. In sweep or dynamic coverage on the other hand the mobile agents are tasked with maximizing a constantly changing objective, resulting in potentially continuous motion of the agents [4–6].

Several aspects of the area coverage problem have been studied over the years, including the effect of robot dynamics [7–9], communication constraints among agents [10–12], complex

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

Theoretical and Experimental Collaborative Area

**Coverage Schemes Using Mobile Agents**

Coverage Schemes Using Mobile Agents

Sotiris Papatheodorou and Anthony Tzes

Sotiris Papatheodorou and Anthony Tzes

http://dx.doi.org/10.5772/intechopen.78940

Abstract

suggested control laws.

cooperative control

1. Introduction

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

#### **Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents** Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

DOI: 10.5772/intechopen.78940

Sotiris Papatheodorou and Anthony Tzes Sotiris Papatheodorou and Anthony Tzes

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.78940

#### Abstract

This chapter is concerned with the development of collaborative control schemes for mobile ground robots for area coverage purposes. The simplest scheme assumes point omnidirectional robots with heterogeneous circular sensing patterns. Using information from their spatial neighbors, each robot (agent) computes its cell relying on the power diagram partitioning. If there is uncertainty in inferring the locations of these robots, the Additively Weighted Guaranteed Voronoi scheme is employed resulting in a rather conservative performance. The aforementioned schemes are enhanced by using a Voronoifree coverage scheme that relies on the knowledge of any arbitrary sensing pattern employed by the agents. Experimental results are offered to highlight the efficiency of the suggested control laws.

Keywords: area coverage, multiagent systems, mobile robot systems, distributed control, cooperative control

#### 1. Introduction

The problem of area coverage is one that has been widely studied in the past decade and consists of the deployment of a sensor-equipped mobile robot team. It is usually categorized as either blanket or sweep coverage. In blanket or static coverage the goal of the robot team is a final static configuration at which an objective function is maximized [1–3]. In sweep or dynamic coverage on the other hand the mobile agents are tasked with maximizing a constantly changing objective, resulting in potentially continuous motion of the agents [4–6].

Several aspects of the area coverage problem have been studied over the years, including the effect of robot dynamics [7–9], communication constraints among agents [10–12], complex

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

non-convex regions [13–15] or guaranteeing collision avoidance among the mobile robots [16, 17]. A wide variety of methods has also been employed for multirobot area coverage such as geometric optimization [18], optimal control [19] or event-triggered control [20]. Due to the widespread adoption of unmanned aerial vehicles (UAVs), they have become a popular platform for area coverage [21–23] since they are usually equipped with visual sensors [24–26].

there is extensive work on the topic [27, 28]. One generalization of this problem arises by allowing each agent to have a different sensing performance, resulting in a heterogeneous team [29–31]. In this chapter we will focus in the coverage of a convex region by a team of

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

http://dx.doi.org/10.5772/intechopen.78940

45

A team of n mobile ground agents is deployed inside the region of interest Ω. Each agent i∈ In is approximated by a point mass located at qi ∈ Ω which is governed by the following kine-

Each agent has been equipped with an omnidirectional sensor with limited sensing radius Ri,

Since the goal of the mobile agent team is the maximization of the covered area using their sensors, while also taking into account the space density function, we define the coverage

1Si

The control objective is the design of a distributed control law for the mobile agents in order to

The first step in designing a distributed control law is finding a method to distribute the computation of the coverage objective H . Due to the heterogeneous sensing performance of the agents, the Voronoi diagram is inadequate for the task as it does not take this information into account. To that extent the power diagram will be used in order to assign a region of responsibility to each agent. In contrast to the Voronoi diagram whose generators are points,

Given a planar region Ω and a set of disks S ¼ f g S1;…; Sn with centers Q ¼ q1;…; qn

<sup>2</sup> <sup>≤</sup> <sup>∥</sup><sup>q</sup> � qj

n o, i <sup>∈</sup>In:

<sup>∥</sup><sup>2</sup> � Rj 2

; ∀j ∈In∖i

radii R ¼ f g R1;…;Rn , the power diagram assigns a convex cell Pi ⊆ Ω to each disk Si

<sup>∥</sup><sup>2</sup> � Ri

The power diagram is a complete tessellation of Ω, thus it holds that

which is allowed to differ among agents, resulting in a circular sensing pattern

H ¼ ð

guarantee monotonic increase of the coverage objective H over time.

Ω

max i ∈In

� � <sup>¼</sup> <sup>q</sup><sup>∈</sup> <sup>Ω</sup> : <sup>∥</sup><sup>q</sup> � qi

<sup>q</sup>\_<sup>i</sup> <sup>¼</sup> ui, q<sup>∈</sup> <sup>Ω</sup>, ui <sup>∈</sup> <sup>R</sup><sup>2</sup> (1)

� �: (2)

ð Þq ϕð Þq dq: (3)

� � and

∥ ≤ Ri

unicycle robots equipped with heterogeneous omnidirectional sensors.

where ui is the velocity control input of the agent.

the generators of the power diagram are disks.

Pið Þ¼ Ω; S q∈ Ω : ∥q � qi

Si qi ;Ri

2.2.1. Problem statement

matic model

objective as

2.2.2. Space partitioning

In this chapter we focus on the blanket coverage problem for a convex region of interest. The techniques outlined are based on geometric optimization principles and result in distributed control schemes. In a distributed control law, each agent uses only local information from its neighboring agents in order to compute its own control input so that a common objective function is maximized. Distributed control laws are highly desirable in multiagent systems because they are easily scalable to large robot teams and because they significantly reduce the computational burden and communication requirements on the agents. Moreover, they are more robust to failures and can adapt to unexpected changes without the need to recompute a new solution as is the case with most centralized control schemes.

The chapter is organized as follows. Section 2.1 contains some mathematical preliminaries which will be relevant throughout the chapter. In Section 2.2 the problem of blanket area coverage in a convex region by a heterogeneous team of agents with omnidirectional sensors is examined. In Section 2.3 the results are extended by taking into account the uncertain positioning of the mobile robots. Section 2.4 presents a tessellation-free method for area coverage by agents with anisotropic sensing patterns. Section 2.5 contains some experimental results and it is followed by concluding remarks.

#### 2. Area coverage using mobile agents

#### 2.1. Mathematical preliminaries

Throughout the chapter we assume a compact, convex region Ω ⊂ R<sup>2</sup> to be covered by the mobile agents and a space density function ϕ : Ω ! Rþ. The space density function is used to encode any a priori information regarding the importance of points in Ω, for example the likelihood that an event may occur at a given point. The boundary of a set S is denoted ∂S and its interior is denoted Int Sð Þ. The set 1f g ;…; n is denoted In. The indicator function 1Sð Þq for a set S and the 2 � 2 rotation matrix Rð Þ θ are respectively

$$\mathbf{1}\_{\mathcal{S}}(q) = \begin{cases} 1 & \text{if } q \in \mathcal{S} \\ 0 & \text{if } q \notin \mathcal{S}' \end{cases} \qquad \qquad \mathbf{R}(\theta) = \begin{bmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{bmatrix}.$$

while the 2 � 2 identity matrix is denoted I2.

#### 2.2. Heterogeneous agents with omnidirectional sensing

One of the simplest variants of the area coverage problem is the case of a team of homogeneous agents with circular sensing footprints. This was one of the first variants to be studied and there is extensive work on the topic [27, 28]. One generalization of this problem arises by allowing each agent to have a different sensing performance, resulting in a heterogeneous team [29–31]. In this chapter we will focus in the coverage of a convex region by a team of unicycle robots equipped with heterogeneous omnidirectional sensors.

#### 2.2.1. Problem statement

non-convex regions [13–15] or guaranteeing collision avoidance among the mobile robots [16, 17]. A wide variety of methods has also been employed for multirobot area coverage such as geometric optimization [18], optimal control [19] or event-triggered control [20]. Due to the widespread adoption of unmanned aerial vehicles (UAVs), they have become a popular platform for area coverage [21–23] since they are usually equipped with visual sensors [24–26]. In this chapter we focus on the blanket coverage problem for a convex region of interest. The techniques outlined are based on geometric optimization principles and result in distributed control schemes. In a distributed control law, each agent uses only local information from its neighboring agents in order to compute its own control input so that a common objective function is maximized. Distributed control laws are highly desirable in multiagent systems because they are easily scalable to large robot teams and because they significantly reduce the computational burden and communication requirements on the agents. Moreover, they are more robust to failures and can adapt to unexpected changes without the need to recompute a

The chapter is organized as follows. Section 2.1 contains some mathematical preliminaries which will be relevant throughout the chapter. In Section 2.2 the problem of blanket area coverage in a convex region by a heterogeneous team of agents with omnidirectional sensors is examined. In Section 2.3 the results are extended by taking into account the uncertain positioning of the mobile robots. Section 2.4 presents a tessellation-free method for area coverage by agents with anisotropic sensing patterns. Section 2.5 contains some experimental

Throughout the chapter we assume a compact, convex region Ω ⊂ R<sup>2</sup> to be covered by the mobile agents and a space density function ϕ : Ω ! Rþ. The space density function is used to encode any a priori information regarding the importance of points in Ω, for example the likelihood that an event may occur at a given point. The boundary of a set S is denoted ∂S and its interior is denoted Int Sð Þ. The set 1f g ;…; n is denoted In. The indicator function 1Sð Þq

, Rð Þ¼ θ

One of the simplest variants of the area coverage problem is the case of a team of homogeneous agents with circular sensing footprints. This was one of the first variants to be studied and

cos θ � sin θ sin θ cos θ

,

" #

new solution as is the case with most centralized control schemes.

results and it is followed by concluding remarks.

2. Area coverage using mobile agents

1Sð Þ¼ q

while the 2 � 2 identity matrix is denoted I2.

for a set S and the 2 � 2 rotation matrix Rð Þ θ are respectively

2.2. Heterogeneous agents with omnidirectional sensing

(

1 if q∈ S 0 if q ∉ S

2.1. Mathematical preliminaries

44 Applications of Mobile Robots

A team of n mobile ground agents is deployed inside the region of interest Ω. Each agent i∈ In is approximated by a point mass located at qi ∈ Ω which is governed by the following kinematic model

$$
\dot{q}\_i = u\_i \quad q \in \Omega, \quad u\_i \in \mathbb{R}^2 \tag{1}
$$

where ui is the velocity control input of the agent.

Each agent has been equipped with an omnidirectional sensor with limited sensing radius Ri, which is allowed to differ among agents, resulting in a circular sensing pattern

$$\mathcal{S}\_i(\boldsymbol{q}\_i, \boldsymbol{R}\_i) = \left\{ \boldsymbol{q} \in \boldsymbol{\Omega} : \quad \|\boldsymbol{q} - \boldsymbol{q}\_i\| \le \, \mathcal{R}\_i \right\}. \tag{2}$$

Since the goal of the mobile agent team is the maximization of the covered area using their sensors, while also taking into account the space density function, we define the coverage objective as

$$\mathcal{HP} = \bigcap\_{\substack{i \in I\_n \\ \Omega}} \max\_{\mathbf{1}\_{\mathcal{S}\_i}(q) \; | \; \phi(q) \; | \; dq. \tag{3}$$

The control objective is the design of a distributed control law for the mobile agents in order to guarantee monotonic increase of the coverage objective H over time.

#### 2.2.2. Space partitioning

The first step in designing a distributed control law is finding a method to distribute the computation of the coverage objective H . Due to the heterogeneous sensing performance of the agents, the Voronoi diagram is inadequate for the task as it does not take this information into account. To that extent the power diagram will be used in order to assign a region of responsibility to each agent. In contrast to the Voronoi diagram whose generators are points, the generators of the power diagram are disks.

Given a planar region Ω and a set of disks S ¼ f g S1;…; Sn with centers Q ¼ q1;…; qn � � and radii R ¼ f g R1;…;Rn , the power diagram assigns a convex cell Pi ⊆ Ω to each disk Si

$$P\_i(\Omega, \mathbb{S}) = \left\{ q \in \Omega : \left\| q - q\_i \right\|^2 - R\_i^{-2} \le \left\| q - q\_j \right\|^2 - R\_j^{-2}, \ \forall j \in I\_n \backslash i \right\}, \ i \in I\_n.$$

The power diagram is a complete tessellation of Ω, thus it holds that

$$\mathfrak{Q} = \bigcup\_{i \in I\_n} P\_{i\prime} \qquad \operatorname{Int}(P\_i) \cap \operatorname{Int}(P\_{\slash}) = \emptyset, \quad \forall i \neq j.$$

A notable property of power diagrams is their duality with power-weighted Delaunay graphs. It has been shown that in order to compute the power cell Pi of point qi , only the powerweighted Delaunay neighbors Ni of point qi need to be considered. The power-weighted Delaunay neighbors of agent i have the property that

$$N\_i = \{ j \in I\_n \: \vert \: i: P\_i \cap P\_j \neq \emptyset \},\tag{4}$$

where α<sup>i</sup> is a positive constant and ni is the outward unit normal vector on ∂PR

<sup>q</sup>\_<sup>i</sup> <sup>¼</sup> <sup>X</sup> i∈ In

is evaluated as follows

<sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup><sup>q</sup> <sup>¼</sup> <sup>∂</sup>

∂qi ð

<sup>i</sup> ni <sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup><sup>q</sup> <sup>þ</sup> <sup>X</sup>

, q∈∂PR

boundary with the r-limited cell of some neighboring agent j. This decomposition is presented

<sup>i</sup> ∩ P<sup>R</sup>

<sup>i</sup> into disjoint sets and corresponding normal vectors.

PR i

Since only the cells of power-weighted Delaunay neighbors of agent i are affected by its

<sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup><sup>q</sup> <sup>þ</sup><sup>X</sup>

¼ 0, by using the Leibniz integral rule [32], the previous equation becomes

υi

ð

∂P<sup>R</sup> j

j ∈ Ni

<sup>i</sup> can be decomposed into three disjoint sets ∂PR

Proof. We start by evaluating the time derivative of the objective using the agent dynamics (1)

ui: By selecting the control law ui ¼ α<sup>i</sup>

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

j6¼i

∂ ∂qi ð

<sup>j</sup> nj ϕð Þq dq

<sup>j</sup> and ni is the outward unit normal vector on

<sup>i</sup> ∩ ∂Si denotes part of the r-limited cell's boundary

<sup>i</sup> ∩ ∂PR

<sup>j</sup> are shown in solid green, red, and blue lines,

<sup>i</sup> <sup>¼</sup> <sup>∂</sup>PR

<sup>i</sup> ∩ ∂Ω denotes the common

<sup>j</sup> denotes the common

<sup>i</sup> ∩ ∂Si � � ∪

PR j

ϕð Þq dq:

∂H ∂qi

trajectories that result in monotonic increase of the coverage objective (6).

∂H ∂qi

we can guarantee monotonic increase of the coverage objective.

ð

PR i

∂P<sup>R</sup> i

υi

that is also part of the boundary of the agent's sensing disk, ∂P<sup>R</sup>

boundary between the r-limited cell and the region, while ∂P<sup>R</sup>

<sup>i</sup> ∩ ∂Ω and ∂P<sup>R</sup>

<sup>j</sup> <sup>¼</sup> <sup>∂</sup><sup>q</sup> ∂qi

we get <sup>∂</sup><sup>H</sup>

<sup>∂</sup><sup>t</sup> <sup>¼</sup> <sup>X</sup> i∈ In

The partial derivative <sup>∂</sup><sup>H</sup>

movement and <sup>∂</sup>ϕð Þ<sup>q</sup>

where υ<sup>i</sup>

∂P<sup>R</sup>

∂P<sup>R</sup>

<sup>i</sup> <sup>∩</sup> <sup>∂</sup><sup>Ω</sup> � � ∪ ∪

in Figure 1 where ∂PR

Figure 1. Decomposition of ∂PR

respectively.

∂H ∂qi ∂qi <sup>∂</sup><sup>t</sup> <sup>¼</sup> <sup>X</sup> i ∈In

∂H ∂qi

∂qi

<sup>i</sup> . The boundary ∂PR

j∈ Ni

∂qi

¼ ∂ ∂qi X i ∈In

> ∂H ∂qi ¼ ð

<sup>j</sup> is the Jacobian matrix υ<sup>i</sup>

∂P<sup>R</sup> <sup>i</sup> ∩ ∂P<sup>R</sup> j � � � �, where <sup>∂</sup>P<sup>R</sup>

<sup>i</sup> ∩ ∂Si, ∂P<sup>R</sup>

<sup>i</sup> , leading the agents to

http://dx.doi.org/10.5772/intechopen.78940

∂H ∂qi

, α<sup>i</sup> > 0,

47

By using the previous definition, the power diagram can be formulated as

$$P\_i(\Omega, \mathbb{Q}) = \left\{ q \in \Omega \, : \, \left\| q - q\_i \right\|^2 - R\_i^2 \le \left\| q - q\_j \right\|^2 - R\_j^2, \,\, \forall j \in \mathcal{N}\_i \right\}, \; i \in I\_n. \tag{5}$$

Since it holds that Ni ⊆ In, each agent i requires information only from its power-weighted Delaunay neighbors in order to compute its own power cell Pi, thus rendering the computation of the power diagram distributed.

Remark 2.1. When the agents' sensing radii are equal, i.e., Ri ¼ Rj, ∀i, j, the power diagram converges to the Voronoi diagram. In that case the computation of the cell of agent i requires information only from the Delaunay neighbors of agent i. Thus the power diagram can be also utilized in the case of agents with homogeneous sensing performance.

For any two agents i and j with Si ∩ Sj 6¼ ; it holds that Si∖Pi ∈ Pj ∩ Sj and Sj∖Pj ∈Pi ∩ Si due to the properties of the power diagram. Thus if a part of some agent i 's sensing pattern is inside the cell of some other agent j, then that part is guaranteed to be sensed by j. Consequently, we define the r-limited power cell of agent i as P<sup>R</sup> <sup>i</sup> ¼ Pi ∩ Si. Thus by utilizing the power diagram, the coverage objective H can be computed as follows

$$\mathcal{A}^{\mathcal{C}} = \sum\_{i \in I\_n} \int\_{P\_i^{\mathbb{R}}} \phi(q) \, \, dq. \tag{6}$$

Since H can be written as a sum of integrals over r-limited power cells and since an agent can compute its own power cell using information just from its power-weighted Delaunay neighbors, the computation of H is distributed.

#### 2.2.3. Control law formulation

Having found a partitioning scheme that allows distributed computation of the coverage objective H , what is left is the derivation of a distributed control law for its maximization.

Theorem 2.1. For a team of mobile ground agents with kinematics (1), sensing performance (2) and using the power diagram partitioning (5), the control law

$$\mu\_i = \alpha\_i \int\_{\partial P\_i^k \cap \partial \mathbb{S}\_i} n\_i \, \phi(q) \, \text{d}q \,\tag{7}$$

where α<sup>i</sup> is a positive constant and ni is the outward unit normal vector on ∂PR <sup>i</sup> , leading the agents to trajectories that result in monotonic increase of the coverage objective (6).

Proof. We start by evaluating the time derivative of the objective using the agent dynamics (1) we get <sup>∂</sup><sup>H</sup> <sup>∂</sup><sup>t</sup> <sup>¼</sup> <sup>X</sup> i∈ In ∂H ∂qi ∂qi <sup>∂</sup><sup>t</sup> <sup>¼</sup> <sup>X</sup> i ∈In ∂H ∂qi q\_ <sup>i</sup> <sup>¼</sup> <sup>X</sup> i∈ In ∂H ∂qi ui: By selecting the control law ui ¼ α<sup>i</sup> ∂H ∂qi , α<sup>i</sup> > 0,

we can guarantee monotonic increase of the coverage objective.

The partial derivative <sup>∂</sup><sup>H</sup> ∂qi is evaluated as follows

Ω ¼ ⋃ i∈In

Delaunay neighbors of agent i have the property that

Pið Þ¼ Ω; Q q∈ Ω : ∥q � qi

of the power diagram distributed.

46 Applications of Mobile Robots

with homogeneous sensing performance.

define the r-limited power cell of agent i as P<sup>R</sup>

bors, the computation of H is distributed.

using the power diagram partitioning (5), the control law

2.2.3. Control law formulation

the coverage objective H can be computed as follows

Pi, Int Pð Þ<sup>i</sup> ∩ Int Pj

It has been shown that in order to compute the power cell Pi of point qi

By using the previous definition, the power diagram can be formulated as

<sup>∥</sup><sup>2</sup> � Ri

A notable property of power diagrams is their duality with power-weighted Delaunay graphs.

weighted Delaunay neighbors Ni of point qi need to be considered. The power-weighted

<sup>2</sup> <sup>≤</sup> <sup>∥</sup><sup>q</sup> � qj

n o

Since it holds that Ni ⊆ In, each agent i requires information only from its power-weighted Delaunay neighbors in order to compute its own power cell Pi, thus rendering the computation

Remark 2.1. When the agents' sensing radii are equal, i.e., Ri ¼ Rj, ∀i, j, the power diagram converges to the Voronoi diagram. In that case the computation of the cell of agent i requires information only from the Delaunay neighbors of agent i. Thus the power diagram can be also utilized in the case of agents

For any two agents i and j with Si ∩ Sj 6¼ ; it holds that Si∖Pi ∈ Pj ∩ Sj and Sj∖Pj ∈Pi ∩ Si due to the properties of the power diagram. Thus if a part of some agent i 's sensing pattern is inside the cell of some other agent j, then that part is guaranteed to be sensed by j. Consequently, we

ð

PR i

Since H can be written as a sum of integrals over r-limited power cells and since an agent can compute its own power cell using information just from its power-weighted Delaunay neigh-

Having found a partitioning scheme that allows distributed computation of the coverage objective H , what is left is the derivation of a distributed control law for its maximization.

Theorem 2.1. For a team of mobile ground agents with kinematics (1), sensing performance (2) and

ð

∂PR <sup>i</sup> ∩ ∂Si

ui ¼ α<sup>i</sup>

<sup>H</sup> <sup>¼</sup> <sup>X</sup> i ∈In

� � ¼ ;, <sup>∀</sup><sup>i</sup> 6¼ <sup>j</sup>:

Ni <sup>¼</sup> <sup>j</sup><sup>∈</sup> In∖<sup>i</sup> : Pi <sup>∩</sup> Pj 6¼ ; � �, (4)

; ∀j ∈ Ni

<sup>i</sup> ¼ Pi ∩ Si. Thus by utilizing the power diagram,

ϕð Þq dq: (6)

ni ϕð Þq dq, (7)

<sup>∥</sup><sup>2</sup> � Rj 2 , only the power-

, i∈ In: (5)

$$\frac{\partial \mathcal{A}^{\boldsymbol{\varrho}}}{\partial q\_{i}} = \frac{\partial}{\partial q\_{i}} \sum\_{i \in I\_{n}} \int\_{P\_{i}^{\mathbb{R}}} \phi(q) \cdot \mathbf{d}q = \frac{\partial}{\partial q\_{i}} \int\_{P\_{i}^{\mathbb{R}}} \phi(q) \cdot \mathbf{d}q \; \; \; + \sum\_{j \neq i} \frac{\partial}{\partial q\_{i}} \int\_{P\_{j}^{\mathbb{R}}} \phi(q) \cdot \mathbf{d}q \; \; \!$$

Since only the cells of power-weighted Delaunay neighbors of agent i are affected by its movement and <sup>∂</sup>ϕð Þ<sup>q</sup> ∂qi ¼ 0, by using the Leibniz integral rule [32], the previous equation becomes

$$\frac{\partial \mathcal{H}}{\partial \boldsymbol{\eta}\_{i}} = \int\_{\partial P\_{i}^{\mathbb{R}}} \boldsymbol{\upsilon}\_{i}^{i} \, \boldsymbol{n}\_{i} \, \phi(\boldsymbol{q}) \, \, \mathrm{d}\boldsymbol{q} \, \, \left. + \sum\_{j \in N\_{i}} \int\_{\partial P\_{j}^{\mathbb{R}}} \boldsymbol{\upsilon}\_{j}^{i} \, \boldsymbol{n}\_{j} \, \phi(\boldsymbol{q}) \, \, \mathrm{d}\boldsymbol{q} \, \, \right|$$

where υ<sup>i</sup> <sup>j</sup> is the Jacobian matrix υ<sup>i</sup> <sup>j</sup> <sup>¼</sup> <sup>∂</sup><sup>q</sup> ∂qi , q∈∂PR <sup>j</sup> and ni is the outward unit normal vector on ∂P<sup>R</sup> <sup>i</sup> . The boundary ∂PR <sup>i</sup> can be decomposed into three disjoint sets ∂PR <sup>i</sup> <sup>¼</sup> <sup>∂</sup>PR <sup>i</sup> ∩ ∂Si � � ∪ ∂P<sup>R</sup> <sup>i</sup> <sup>∩</sup> <sup>∂</sup><sup>Ω</sup> � � ∪ ∪ j∈ Ni ∂P<sup>R</sup> <sup>i</sup> ∩ ∂P<sup>R</sup> j � � � �, where <sup>∂</sup>P<sup>R</sup> <sup>i</sup> ∩ ∂Si denotes part of the r-limited cell's boundary that is also part of the boundary of the agent's sensing disk, ∂P<sup>R</sup> <sup>i</sup> ∩ ∂Ω denotes the common boundary between the r-limited cell and the region, while ∂P<sup>R</sup> <sup>i</sup> ∩ ∂PR <sup>j</sup> denotes the common boundary with the r-limited cell of some neighboring agent j. This decomposition is presented in Figure 1 where ∂PR <sup>i</sup> ∩ ∂Si, ∂P<sup>R</sup> <sup>i</sup> ∩ ∂Ω and ∂P<sup>R</sup> <sup>i</sup> ∩ P<sup>R</sup> <sup>j</sup> are shown in solid green, red, and blue lines, respectively.

Figure 1. Decomposition of ∂PR <sup>i</sup> into disjoint sets and corresponding normal vectors.

Since the region Ω is assumed to be static, it holds that υ<sup>i</sup> <sup>i</sup> <sup>¼</sup> <sup>0</sup>, <sup>∀</sup>q<sup>∈</sup> <sup>∂</sup>P<sup>R</sup> <sup>i</sup> ∩ ∂Ω. In addition, since q∈∂PR <sup>i</sup> ∩ ∂Si are points on a circle with center qi , it holds that υ<sup>i</sup> <sup>i</sup> <sup>¼</sup> <sup>I</sup>2, <sup>∀</sup>q∈∂PR <sup>i</sup> ∩ ∂Si. Finally, PR j is only affected by the movement of agent i at the common boundary ∂P<sup>R</sup> <sup>i</sup> ∩ ∂PR <sup>j</sup> , resulting in the expression

metric indicates to what extent the agents cover the region Ω, with high values of H <sup>r</sup>

coverage objective H as a percentage of the agents' maximum possible covered area which is only meaningful in the case where ϕð Þ¼ q 1, ∀q∈ Ω. This metric indicates to what extent the agents' sensors are utilized, with high values of H <sup>a</sup> indicating that the agents' sensors are utilized close to their full capabilities. Low values of H <sup>a</sup> simultaneously with high values of H <sup>r</sup> indicate an overabundance of agents given the size of the current region Ω. The two

Figure 2d shows H <sup>a</sup> in solid blue and H <sup>r</sup> in dashed red with their final values being 90.0 and

The inherent uncertainty in all localization systems' measurements can often create unexpected problems in algorithms designed with precise localization in mind. Consequently algorithms robust to positioning errors have been sought for the area coverage problem [33, 34]. This section presents an extension to the control law presented in [35] which allows for teams of agents with

The agents' kinematic model is described by (1) and their sensing performance by (2). Due to the localization systems' inherent inaccuracy, we assume that qi is the agent's position as reported by the localization system, while ri is a known upper bound on the localization error.

which is a circular disk that contains all possible positions of agent i given its reported position

In order to take into account the agents' positioning uncertainty, we define for each agent the

which is the region that is guaranteed to be sensed by agent i given all of its possible positions within its positioning uncertainty region. Given the fact that both Si and Ui are disks, the above

∀q∈ Ui

∥ ≤ Rg

<sup>i</sup> ¼ Ri � ri � �: (11)

� � <sup>¼</sup> <sup>q</sup><sup>∈</sup> <sup>R</sup><sup>2</sup> : <sup>∥</sup><sup>q</sup> � qi

2.3. Heterogeneous agents with omnidirectional sensing under positioning uncertainty

, <sup>H</sup> <sup>a</sup> <sup>¼</sup> <sup>100</sup> <sup>H</sup>

X i∈In

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

∥ ≤ ri

� �, (9)

Sið Þ q; Ri , (10)

ð

Si dq , is the value of the

49

http://dx.doi.org/10.5772/intechopen.78940

: (8)

corresponding to high coverage over Ω. The second metric, denoted H <sup>a</sup>

<sup>H</sup> <sup>r</sup> <sup>¼</sup> <sup>100</sup> <sup>H</sup> ð

Thus we define the positioning uncertainty region Ui as follows

Ui qi ;ri

<sup>i</sup> as

Sg <sup>i</sup> qi ;ri;Ri � � <sup>¼</sup> <sup>⋂</sup>

� � <sup>¼</sup> <sup>q</sup><sup>∈</sup> <sup>R</sup><sup>2</sup> : <sup>∥</sup><sup>q</sup> � qi

qi and positioning uncertainty upper bound ri.

Sg <sup>i</sup> qi ;ri;Ri

guaranteed sensed region Sg

expression can be simplified into

Ω

ϕð Þq dq

88.9% respectively, indicating that the final agent configuration is an efficient one.

metrics are more formally defined as

heterogeneous sensing performance.

2.3.1. Agent model

$$\frac{\partial \mathcal{A}^{\boldsymbol{\varrho}}}{\partial q\_{i}} = \int\_{\partial \mathbb{P}^{\mathbb{R}}\_{i} \cap \mathrm{d}\mathbb{S}\_{i}} n\_{i} \, \phi(q) \, \mathrm{d}\boldsymbol{\eta} \, \, + \sum\_{j \in \mathcal{N}\_{i}} \int\_{\partial \mathbb{P}^{\mathbb{R}}\_{i} \cap \mathrm{d}\mathbb{P}^{\mathbb{R}}\_{j}} \boldsymbol{\upsilon}^{i}\_{i} \, n\_{i} \, \phi(q) \, \, \mathrm{d}\boldsymbol{\eta} \, \, \, + \sum\_{j \in \mathcal{N}\_{i}} \int\_{\partial \mathbb{P}^{\mathbb{R}}\_{i} \cap \mathrm{d}\mathbb{P}^{\mathbb{R}}\_{j}} \boldsymbol{\upsilon}^{i}\_{j} \, n\_{j} \, \phi(q) \, \, \, \mathrm{d}\boldsymbol{\eta} \, \, \, \, $$

Since υ<sup>i</sup> <sup>i</sup> <sup>¼</sup> <sup>υ</sup><sup>i</sup> <sup>j</sup> and ni ¼ �nj on the common boundary <sup>∂</sup>PR <sup>i</sup> ∩ ∂P<sup>R</sup> <sup>j</sup> , as shown in Figure 1, the two sums in the previous expression cancel out and by multiplying it with <sup>α</sup><sup>i</sup> we get (7). □

#### 2.2.4. Simulation study I

An indicative simulation is presented in this section. The region Ω is chosen as the convex polygon defined by the vertices with x and y coordinates

$$\begin{aligned} \Omega\_{\mathbf{x}} &= [0.5, 0.5, 0.45, 0.4, -0.46, -0.5, -0.48, -0.34, 0.05], \\ \Omega\_{\mathbf{y}} &= [0.43, 0.2, -0.3, -0.5, -0.44, -0.1, 0.37, 0.47, 0.5] \end{aligned}$$

respectively. The space density function was ϕð Þ¼ q 1, ∀q∈ Ω. A team of eight agents with heterogeneous sensing radii is deployed inside the region.

The initial and final agent configurations are shown in Figure 2a and c respectively where the agent positions are marked by black dots, the boundaries of their sensing disks are shown as dashed black lines, the boundaries of their cells are marked by solid black lines while their interiors are filled in color. The agent trajectories are shown in Figure 2b with the initial positions marked by dots and the final positions by circles. It is observed that the agents are successfully deployed inside the region, increasing the covered area in the process. In order to provide a more objective measure of the agents' performance, two different metrics are used. The first, denoted H <sup>r</sup> , is the value of the coverage objective H as a percentage of the objective over the whole region which in the case where ϕð Þ¼ q 1, ∀q∈ Ω it is equal to the area of Ω. This

Figure 2. Simulation study I: (a) initial configuration, (b) agent trajectories, (c) final configuration and (d) evolution of the coverage objective over time.

metric indicates to what extent the agents cover the region Ω, with high values of H <sup>r</sup> corresponding to high coverage over Ω. The second metric, denoted H <sup>a</sup> , is the value of the coverage objective H as a percentage of the agents' maximum possible covered area which is only meaningful in the case where ϕð Þ¼ q 1, ∀q∈ Ω. This metric indicates to what extent the agents' sensors are utilized, with high values of H <sup>a</sup> indicating that the agents' sensors are utilized close to their full capabilities. Low values of H <sup>a</sup> simultaneously with high values of H <sup>r</sup> indicate an overabundance of agents given the size of the current region Ω. The two metrics are more formally defined as

$$\mathcal{H}^{\ell} = 100 \frac{\mathcal{H}^{\ell}}{\int\_{\Omega} \phi(q) \mathbf{d}q}, \qquad \mathcal{H}^{a} = 100 \frac{\mathcal{H}^{\ell}}{\sum\_{i \in I\_{n}} \Big| \mathbf{d}q}. \tag{8}$$

Figure 2d shows H <sup>a</sup> in solid blue and H <sup>r</sup> in dashed red with their final values being 90.0 and 88.9% respectively, indicating that the final agent configuration is an efficient one.

#### 2.3. Heterogeneous agents with omnidirectional sensing under positioning uncertainty

The inherent uncertainty in all localization systems' measurements can often create unexpected problems in algorithms designed with precise localization in mind. Consequently algorithms robust to positioning errors have been sought for the area coverage problem [33, 34]. This section presents an extension to the control law presented in [35] which allows for teams of agents with heterogeneous sensing performance.

#### 2.3.1. Agent model

Since the region Ω is assumed to be static, it holds that υ<sup>i</sup>

ni <sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup><sup>q</sup> <sup>þ</sup> <sup>X</sup>

polygon defined by the vertices with x and y coordinates

heterogeneous sensing radii is deployed inside the region.

is only affected by the movement of agent i at the common boundary ∂P<sup>R</sup>

j∈ Ni

<sup>j</sup> and ni ¼ �nj on the common boundary <sup>∂</sup>PR

ð

υi

sums in the previous expression cancel out and by multiplying it with <sup>α</sup><sup>i</sup> we get (7). □

An indicative simulation is presented in this section. The region Ω is chosen as the convex

Ω<sup>x</sup> ¼ ½ � 0:5; 0:5; 0:45; 0:4; �0:46; �0:5; �0:48; �0:34; 0:05 , Ω<sup>y</sup> ¼ ½ � 0:43; 0:2; �0:3; �0:5; �0:44; �0:1; 0:37; 0:47; 0:5

respectively. The space density function was ϕð Þ¼ q 1, ∀q∈ Ω. A team of eight agents with

The initial and final agent configurations are shown in Figure 2a and c respectively where the agent positions are marked by black dots, the boundaries of their sensing disks are shown as dashed black lines, the boundaries of their cells are marked by solid black lines while their interiors are filled in color. The agent trajectories are shown in Figure 2b with the initial positions marked by dots and the final positions by circles. It is observed that the agents are successfully deployed inside the region, increasing the covered area in the process. In order to provide a more objective measure of the agents' performance, two different metrics are used.

over the whole region which in the case where ϕð Þ¼ q 1, ∀q∈ Ω it is equal to the area of Ω. This

Figure 2. Simulation study I: (a) initial configuration, (b) agent trajectories, (c) final configuration and (d) evolution of the

∂P<sup>R</sup> <sup>i</sup> <sup>∩</sup> <sup>∂</sup>P<sup>R</sup> j

<sup>i</sup> ∩ ∂Si are points on a circle with center qi

q∈∂PR

expression

Since υ<sup>i</sup>

∂H ∂qi ¼

48 Applications of Mobile Robots

<sup>i</sup> <sup>¼</sup> <sup>υ</sup><sup>i</sup>

2.2.4. Simulation study I

The first, denoted H <sup>r</sup>

coverage objective over time.

ð

∂P<sup>R</sup> <sup>i</sup> ∩ ∂Si <sup>i</sup> <sup>¼</sup> <sup>0</sup>, <sup>∀</sup>q<sup>∈</sup> <sup>∂</sup>P<sup>R</sup>

j∈ Ni

<sup>i</sup> <sup>¼</sup> <sup>I</sup>2, <sup>∀</sup>q∈∂PR

ð

∂P<sup>R</sup> <sup>i</sup> <sup>∩</sup> <sup>∂</sup>P<sup>R</sup> j

<sup>i</sup> ∩ ∂PR

υi

<sup>j</sup> , as shown in Figure 1, the two

, it holds that υ<sup>i</sup>

<sup>i</sup> ni <sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup><sup>q</sup> <sup>þ</sup> <sup>X</sup>

<sup>i</sup> ∩ ∂P<sup>R</sup>

, is the value of the coverage objective H as a percentage of the objective

<sup>i</sup> ∩ ∂Ω. In addition, since

<sup>j</sup> nj ϕð Þq dq:

<sup>i</sup> ∩ ∂Si. Finally, PR

<sup>j</sup> , resulting in the

j

The agents' kinematic model is described by (1) and their sensing performance by (2). Due to the localization systems' inherent inaccuracy, we assume that qi is the agent's position as reported by the localization system, while ri is a known upper bound on the localization error. Thus we define the positioning uncertainty region Ui as follows

$$\mathcal{U}l\_i(q\_i, r\_i) = \{q \in \mathbb{R}^2 \, : \, \|q - q\_i\| \le r\_i\},\tag{9}$$

which is a circular disk that contains all possible positions of agent i given its reported position qi and positioning uncertainty upper bound ri.

In order to take into account the agents' positioning uncertainty, we define for each agent the guaranteed sensed region Sg <sup>i</sup> as

$$S\_i^{\mathbb{S}}(q\_i, r\_i, R\_i) = \bigcap\_{\forall q \in \mathcal{U}\_i} S\_i(q, R\_i), \tag{10}$$

which is the region that is guaranteed to be sensed by agent i given all of its possible positions within its positioning uncertainty region. Given the fact that both Si and Ui are disks, the above expression can be simplified into

$$S\_i^{\mathcal{S}}(\boldsymbol{\eta}\_i, r\_i, \boldsymbol{R}\_i) = \{ \boldsymbol{\eta} \in \mathbb{R}^2 \, : \, \|\boldsymbol{\eta} - \boldsymbol{\eta}\_i\| \, \le \boldsymbol{R}\_i^{\mathcal{S}} = \boldsymbol{R}\_i - r\_i \}. \tag{11}$$

#### 2.3.2. Space partitioning and problem statement

In order to take into account the agents' positioning uncertainty as well as their heterogeneous sensing capabilities, the Additively Weighted Guaranteed Voronoi (AWGV) diagram is employed. The AWGV is an extension of the Guaranteed Voronoi (GV) diagram [36] that incorporates additive weights.

Given a planar region Ω, a set of uncertain regions U ¼ f g U1;…; Un and a set of weights Rg <sup>¼</sup> Rg 1;…;R<sup>g</sup> n � �, the AWGV diagram assigns a convex cell Gi ⊆ Ω to each region-weight pair Ui;Rg i � � as follows

$$\mathcal{G}\_{l}(\Omega, \mathcal{U}, \mathbb{R}^{\mathcal{S}}) = \left\{ q \in \Omega : \max\_{q \in \mathcal{U}\_{i}} \|q - q\_{i}\| - R\_{i}^{\mathcal{S}} \le \min\_{q \in \mathcal{U}\_{j}} \|q - q\_{j}\| - R\_{j}^{\mathcal{S}}, \ \forall j \in I\_{n} \,\backslash i \right\}, \quad i \in I\_{n} \dots$$

Contrary to the Voronoi diagram, the AWGV diagram is not a complete tessellation of the region Ω since a part of Ω is left unassigned. This part is called the neutral region O and it holds that

$$
\Omega = \mathcal{O} \cup \bigcup\_{i \in I\_{\mathbb{n}}} \mathcal{G}\_{i}. \tag{12}
$$

which is the area of the region that is guaranteed to be closest to and at the same time sensed by each agent. Since H is a sum of integrals over r-limited AWGV cells and since an agent can compute its own AWGV cell using information just from the agents in Ni, the computation of

Since the computation of the coverage objective H is distributed due to the AWGV partitioning, what is left is the derivation of a distributed control law for its maximization.

Theorem 2.2. For a team of mobile ground agents with kinematics (1), sensing performance (2),

μi

Proof. We start by evaluating the time derivative of the objective using the agent dynamics (1)

<sup>i</sup> ni ϕð Þq dq þ

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

<sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup><sup>q</sup> <sup>þ</sup><sup>X</sup>

j6¼i

∂ ∂qi ð

<sup>j</sup> nj ϕð Þq dq

GR j

ð

μi

http://dx.doi.org/10.5772/intechopen.78940

<sup>j</sup> nj ϕð Þq dq

<sup>i</sup> , leads the agent to trajectories

∂H ∂qi

ϕð Þq dq:

, α<sup>i</sup> > 0, we can

(15)

51

∂GR <sup>j</sup> ∩ ∂Hji

positioning uncertainty (9) and using the AWGV partitioning (13), the control scheme

ð

∂G<sup>R</sup> <sup>i</sup> ∩ ∂Hij

X j ∈ Ni

where α<sup>i</sup> is a positive constant, ni the outward unit normal vector on ∂G<sup>R</sup>

as in the proof of Theorem 2.1 and by selecting the control law ui ¼ α<sup>i</sup>

is evaluated as follows

<sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup><sup>q</sup> <sup>¼</sup> <sup>∂</sup>

¼ 0, the previous equation becomes

μi <sup>j</sup> <sup>¼</sup> <sup>∂</sup><sup>q</sup> ∂qi

∂qi ð

<sup>i</sup> ni <sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup><sup>q</sup> <sup>þ</sup> <sup>X</sup>

GR i

Since only the cells of additively weighted Delaunay neighbors of agent i are affected by its

j ∈ Ni

, q∈∂GR j

ð

μi

∂GR j

that result in monotonic increase of the coverage objective (14).

guarantee monotonic increase of the coverage objective.

ð

GR i

∂GR i

μi

∂qi

¼ ∂ ∂qi X i ∈In

> ∂H ∂qi ¼ ð

H is distributed.

ui ¼ α<sup>i</sup>

2.3.3. Control law formulation

ð

ni ϕð Þq dq þ α<sup>i</sup>

∂G<sup>R</sup> <sup>i</sup> <sup>∩</sup> <sup>∂</sup>Sg i

The partial derivative <sup>∂</sup><sup>H</sup>

movement and <sup>∂</sup>ϕð Þ<sup>q</sup>

where μ<sup>i</sup>

∂H ∂qi

∂qi

<sup>j</sup> is the Jacobian matrix

A notable property of AWGV diagrams is their duality with additively weighted Delaunay graphs. It has been shown that in order to compute the AWGV cell Gi of the region-weight pair Ui;R<sup>g</sup> i � �, only the additively weighted Delaunay neighbors Ni of Ui;R<sup>g</sup> i � � need to be considered. By using the previous definition, the Voronoi diagram can be formulated as

$$G\_i(\Omega, \mathcal{U}, \mathbb{R}^{\mathcal{S}}) = \left\{ \boldsymbol{\eta} \in \Omega : \max\_{\boldsymbol{\eta} \in \mathcal{U}\_i} \| \boldsymbol{\eta} - \boldsymbol{\eta}\_i \| - \mathbb{R}\_i^{\mathcal{S}} \le \min\_{\boldsymbol{\eta} \in \mathcal{U}\_j} \| \boldsymbol{\eta} - \boldsymbol{\eta}\_j \| - \mathbb{R}\_j^{\mathcal{S}}, \ \forall j \in \mathcal{N}\_i \, \middle\vert \, i \in \mathcal{I}\_n. \tag{13} \right\}$$

Since it holds that Ni ⊆In, each agent i requires information only from its additively weighted Delaunay neighbors in order to compute its own AWGV cell Gi, thus rendering the computation of the AWGV diagram distributed.

The previous definition of the AWGV can be written as Gi ¼ ⋂ j ∈ Ni Hij, i∈ In, where Hij ¼ q∈ Ω : ∥q � qj ∥ � ∥q � qi <sup>∥</sup> <sup>≥</sup> <sup>þ</sup> ri <sup>þ</sup> rj � Rg <sup>i</sup> <sup>þ</sup> Rg j n o. Thus the boundary <sup>∂</sup>Hij is one branch of a hyperbola with foci located at qi and qi and semi-major axis aij <sup>¼</sup> riþrj�Rg <sup>i</sup> <sup>þ</sup>R<sup>g</sup> j <sup>2</sup> . Since the quantity aij may be either positive or negative, ∂Hij may correspond to the 'East' or 'West' branch of the hyperbola, which results in the region Hij being convex or non-convex respectively.

We define the r-limited AWGV cell of agent i as GR <sup>i</sup> <sup>¼</sup> Gi <sup>∩</sup> Sg <sup>i</sup> . We now define the coverage objective as

$$\mathcal{AC} = \sum\_{i \in I\_n} \int\_{G\_i^{\mathbb{R}}} \phi(q) \, dq\_{\prime} \tag{14}$$

which is the area of the region that is guaranteed to be closest to and at the same time sensed by each agent. Since H is a sum of integrals over r-limited AWGV cells and since an agent can compute its own AWGV cell using information just from the agents in Ni, the computation of H is distributed.

#### 2.3.3. Control law formulation

2.3.2. Space partitioning and problem statement

incorporates additive weights.

Gi <sup>Ω</sup>; <sup>U</sup>;R<sup>g</sup> ð Þ¼ <sup>q</sup><sup>∈</sup> <sup>Ω</sup> : max

Gi <sup>Ω</sup>; <sup>U</sup>;R<sup>g</sup> ð Þ¼ <sup>q</sup><sup>∈</sup> <sup>Ω</sup> : max

tion of the AWGV diagram distributed.

Hij ¼ q∈ Ω : ∥q � qj

objective as

q∈ Ui

∥q � qi

� �, only the additively weighted Delaunay neighbors Ni of Ui;R<sup>g</sup>

∥q � qi

The previous definition of the AWGV can be written as Gi ¼ ⋂

<sup>∥</sup> <sup>≥</sup> <sup>þ</sup> ri <sup>þ</sup> rj � Rg

of a hyperbola with foci located at qi and qi and semi-major axis aij <sup>¼</sup> riþrj�Rg

<sup>H</sup> <sup>¼</sup> <sup>X</sup> i ∈In

q∈ Ui

∥ � ∥q � qi

We define the r-limited AWGV cell of agent i as GR

n o

ered. By using the previous definition, the Voronoi diagram can be formulated as

<sup>∥</sup> � Rg

1;…;R<sup>g</sup> n

i � � as follows

50 Applications of Mobile Robots

Rg <sup>¼</sup> Rg

Ui;R<sup>g</sup> i

pair Ui;Rg

In order to take into account the agents' positioning uncertainty as well as their heterogeneous sensing capabilities, the Additively Weighted Guaranteed Voronoi (AWGV) diagram is employed. The AWGV is an extension of the Guaranteed Voronoi (GV) diagram [36] that

Given a planar region Ω, a set of uncertain regions U ¼ f g U1;…; Un and a set of weights

Contrary to the Voronoi diagram, the AWGV diagram is not a complete tessellation of the region Ω since a part of Ω is left unassigned. This part is called the neutral region O and it holds that

A notable property of AWGV diagrams is their duality with additively weighted Delaunay graphs. It has been shown that in order to compute the AWGV cell Gi of the region-weight pair

> <sup>i</sup> ≤ min q ∈ Uj

Since it holds that Ni ⊆In, each agent i requires information only from its additively weighted Delaunay neighbors in order to compute its own AWGV cell Gi, thus rendering the computa-

quantity aij may be either positive or negative, ∂Hij may correspond to the 'East' or 'West' branch of the hyperbola, which results in the region Hij being convex or non-convex respectively.

ð

GR i

<sup>i</sup> <sup>þ</sup> Rg j

<sup>i</sup> <sup>¼</sup> Gi <sup>∩</sup> Sg

� �

∥q � qj

<sup>∥</sup> � Rg

Ω ¼ O ∪ ⋃

<sup>∥</sup> � Rg

� �, the AWGV diagram assigns a convex cell Gi ⊆ Ω to each region-weight

<sup>i</sup> ≤ min q ∈ Uj

i∈ In

� �

∥q � qj

<sup>∥</sup> � <sup>R</sup><sup>g</sup>

<sup>j</sup> ; ∀j∈ In∖i

Gi: (12)

i

<sup>j</sup> ; ∀j∈ Ni∖i

j ∈ Ni

ϕð Þq dq, (14)

. Thus the boundary ∂Hij is one branch

� � need to be consid-

, i∈ In: (13)

Hij, i∈ In, where

<sup>i</sup> <sup>þ</sup>R<sup>g</sup> j <sup>2</sup> . Since the

<sup>i</sup> . We now define the coverage

, i∈ In:

Since the computation of the coverage objective H is distributed due to the AWGV partitioning, what is left is the derivation of a distributed control law for its maximization.

Theorem 2.2. For a team of mobile ground agents with kinematics (1), sensing performance (2), positioning uncertainty (9) and using the AWGV partitioning (13), the control scheme

$$\begin{aligned} \rho\_i u\_i &= a\_i \int\_{\partial \mathbb{G}\_i^R \cap \partial \mathbb{S}\_i^S} \phi\_i(q) \, \mathrm{d}q \, \, + \, a\_i \sum\_{j \in N\_i} \left[ \int\_{\partial \mathbb{G}\_i^R \cap \partial \mathbb{H}\_{\bar{q}}} \mu\_i^i \, n\_i \, \phi(q) \, \, \mathrm{d}q \, \, + \, \int\_{\partial \mathbb{G}\_{\bar{\gamma}}^R \cap \partial \mathbb{H}\_{\bar{p}}} \mu\_{\bar{q}}^i \, \phi(q) \, \, \mathrm{d}q \right] \end{aligned} \tag{15}$$

where α<sup>i</sup> is a positive constant, ni the outward unit normal vector on ∂G<sup>R</sup> <sup>i</sup> , leads the agent to trajectories that result in monotonic increase of the coverage objective (14).

Proof. We start by evaluating the time derivative of the objective using the agent dynamics (1) as in the proof of Theorem 2.1 and by selecting the control law ui ¼ α<sup>i</sup> ∂H ∂qi , α<sup>i</sup> > 0, we can guarantee monotonic increase of the coverage objective.

The partial derivative <sup>∂</sup><sup>H</sup> ∂qi is evaluated as follows

$$\frac{\partial \mathcal{M}^{\rho}}{\partial q\_{i}} = \frac{\partial}{\partial q\_{i}} \sum\_{i \in I\_{n}} \int\_{G\_{i}^{\mathbb{R}}} \phi(q) \cdot \mathbf{d}q = \frac{\partial}{\partial q\_{i}} \int\_{G\_{i}^{\mathbb{R}}} \phi(q) \cdot \mathbf{d}q \; \; \; + \sum\_{j \neq i} \frac{\partial}{\partial q\_{i}} \int\_{G\_{j}^{\mathbb{R}}} \phi(q) \cdot \mathbf{d}q \; \; \; \; $$

Since only the cells of additively weighted Delaunay neighbors of agent i are affected by its movement and <sup>∂</sup>ϕð Þ<sup>q</sup> ∂qi ¼ 0, the previous equation becomes

$$\frac{\partial \mathcal{H}}{\partial \boldsymbol{\eta}\_{i}} = \int\_{\partial \mathcal{G}^{\mathbb{R}}\_{i}} \mu^{i}\_{i} \, \boldsymbol{n}\_{i} \, \phi(\boldsymbol{q}) \, \, \mathrm{d}\boldsymbol{q} \, \, \left. + \sum\_{j \in \mathcal{N}\_{i}} \int\_{\partial \mathcal{G}^{\mathbb{R}}\_{j}} \mu^{i}\_{j} \, \boldsymbol{n}\_{j} \, \phi(\boldsymbol{q}) \, \, \mathrm{d}\boldsymbol{q} \, \, \right|$$

where μ<sup>i</sup> <sup>j</sup> is the Jacobian matrix

$$
\mu^i\_j = \frac{\partial q}{\partial q\_i}, \quad q \in \partial G^R\_j
$$

and ni is the outward unit normal vector on ∂G<sup>R</sup> <sup>i</sup> . The boundary ∂G<sup>R</sup> <sup>i</sup> can be decomposed into three disjoint sets as follows

$$
\partial \mathsf{G}\_i^R = \left( \partial \mathsf{G}\_i^R \cap \partial \mathsf{S}\_i^{\emptyset} \right) \cup \left( \partial \mathsf{G}\_i^R \cap \partial \mathsf{O} \right) \cup \left[ \bigcup\_{j \in N\_i} \left( \partial \mathsf{G}\_i^R \cap \partial \mathsf{H}\_{\vec{\eta}} \right) \right], \tag{16}
$$

By using this subset Ω<sup>s</sup>

2.3.5. Simulation study II

tainty to the agents.

coverage objective over time.

control law being implemented is

where ε is an infinitesimally small positive constant.

<sup>H</sup> <sup>r</sup> <sup>¼</sup> <sup>100</sup> <sup>H</sup> ð

Ω

ϕð Þq dq

<sup>i</sup> , constraining agents inside Ω is simpler, since this is equivalent to

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

<sup>i</sup> <sup>∧</sup> qi <sup>þ</sup> <sup>ε</sup>ui <sup>∉</sup> <sup>Ω</sup><sup>s</sup>

i

http://dx.doi.org/10.5772/intechopen.78940

<sup>i</sup> . This is

<sup>i</sup> . Thus the

<sup>i</sup> and

53

(17)

constraining each agent's reported position qi inside its respective subset region <sup>Ω</sup><sup>s</sup>

simultaneously its current control input leads the agent toward the exterior of Ω<sup>s</sup>

<sup>u</sup>~<sup>i</sup> <sup>¼</sup> 0 if qi <sup>∈</sup>∂Ω<sup>s</sup>

ui otherwise �

achieved by stopping an agent if its reported position qi is located on the boundary of <sup>Ω</sup><sup>s</sup>

An indicative simulation is presented in this section. This simulation is identical to the one presented in Section 2.2.4 with the only difference being the addition of positioning uncer-

The initial and final agent configurations are shown in Figure 4a and c respectively where the agent positioning uncertainty regions are shown as black circles, the boundaries of their sensing disks are shown as dashed black lines, the boundaries of their cells are marked by solid black lines while their interiors are filled in color. The agent trajectories are shown in Figure 4b with the initial positions marked by dots and the final positions by circles. It is observed that the agents successfully deploy inside the region, increasing the covered area in the process. In order to provide a more objective measure of the agents' performance, the two metrics described in Section 2.2.4 are used which in the case of uncertainty positioning are more formally defined as

Figure 4d shows H <sup>a</sup> in solid blue and H <sup>r</sup> in dashed red with their final values being 94.0 and 70.0% respectively. In this simulation we observe that although H <sup>a</sup> reaches a high value, this is

Figure 4. Simulation study II: (a) initial configuration, (b) agent trajectories, (c) final configuration and (d) evolution of the

, <sup>H</sup> <sup>a</sup> <sup>¼</sup> <sup>100</sup> <sup>H</sup>

X i∈ In ð

dq :

Sg i

where ∂G<sup>R</sup> <sup>i</sup> <sup>∩</sup> <sup>∂</sup>Sg <sup>i</sup> denotes part of the r-limited cell's boundary that is also part of the boundary of the agent's sensing disk, ∂GR <sup>i</sup> ∩ ∂Ω denotes the common boundary between the r-limited cell and the region, while ∂G<sup>R</sup> <sup>i</sup> ∩ ∂Hij denotes parts of the boundary that consist of hyperbolic arcs induced by some neighboring agent j. This decomposition is presented in Figure 3 where ∂G<sup>R</sup> <sup>i</sup> <sup>∩</sup> <sup>∂</sup>Sg <sup>i</sup> , <sup>∂</sup>G<sup>R</sup> <sup>i</sup> ∩ ∂Ω, ∂G<sup>R</sup> <sup>i</sup> ∩ ∂Hij and ∂G<sup>R</sup> <sup>j</sup> ∩ ∂Hji are shown in solid green, red, blue and purple lines respectively.

Since the region Ω is assumed to be static, it holds that μ<sup>i</sup> <sup>i</sup> <sup>¼</sup> <sup>0</sup>, <sup>∀</sup>q∈∂G<sup>R</sup> <sup>i</sup> ∩ ∂Ω. In addition, since q∈∂GR <sup>i</sup> <sup>∩</sup> <sup>∂</sup>S<sup>g</sup> <sup>i</sup> are points on a circle with center qi , it holds that μ<sup>i</sup> <sup>i</sup> <sup>¼</sup> <sup>I</sup>2, <sup>∀</sup>q∈∂GR <sup>i</sup> <sup>∩</sup> <sup>∂</sup>Sg <sup>i</sup> . Finally, GR j is only affected by the movement of agent i at the induced hyperbolic arc ∂G<sup>R</sup> <sup>j</sup> ∩ ∂Hji and by grouping the hyperbolic arcs in pairs and multiplying by <sup>α</sup><sup>i</sup> we get (15). □

#### 2.3.4. Constraining agents inside the region

When the control law (15) is used, there can be cases where the positioning uncertainty regions of some agent does not remain entirely inside Ω, i.e. it is possible that Ui ∩ Ω 6¼ Ui for some agent i. This has the implication that the control law (15) may lead some agent i outside the region Ω, given the fact that an agent may reside anywhere within its positioning uncertainty region Ui.

In order to avoid such a situation, a subset Ω<sup>s</sup> <sup>i</sup> ⊆ Ω is used instead, instead of the region Ω. This subset Ω<sup>s</sup> <sup>i</sup> is in the general case different among agents due to their differing measures of positioning uncertainty ri . This subset of Ω is computed as the Minkowski difference of Ω with the disk U<sup>0</sup> <sup>i</sup> ð Þ¼ ri f g <sup>q</sup><sup>∈</sup> <sup>Ω</sup> : <sup>∥</sup>q<sup>∥</sup> <sup>≤</sup> ri which is <sup>Ω</sup><sup>s</sup> <sup>i</sup> <sup>¼</sup> <sup>q</sup><sup>∈</sup> <sup>Ω</sup> : <sup>q</sup> <sup>þ</sup> <sup>U</sup><sup>0</sup> <sup>i</sup> <sup>⊆</sup> <sup>Ω</sup> � �, i <sup>∈</sup>In.

Figure 3. Decomposition of ∂GR <sup>i</sup> into disjoint sets and corresponding normal vectors.

By using this subset Ω<sup>s</sup> <sup>i</sup> , constraining agents inside Ω is simpler, since this is equivalent to constraining each agent's reported position qi inside its respective subset region <sup>Ω</sup><sup>s</sup> <sup>i</sup> . This is achieved by stopping an agent if its reported position qi is located on the boundary of <sup>Ω</sup><sup>s</sup> <sup>i</sup> and simultaneously its current control input leads the agent toward the exterior of Ω<sup>s</sup> <sup>i</sup> . Thus the control law being implemented is

$$
\tilde{\mu}\_i = \begin{cases} 0 & \text{if } q\_i \in \partial \Omega\_i^s \land q\_i + \varepsilon \, u\_i \notin \Omega\_i^s \\\ \mu\_i & \text{otherwise} \end{cases} \tag{17}
$$

where ε is an infinitesimally small positive constant.

#### 2.3.5. Simulation study II

and ni is the outward unit normal vector on ∂G<sup>R</sup>

∂G<sup>R</sup>

<sup>i</sup> <sup>¼</sup> <sup>∂</sup>GR

<sup>i</sup> <sup>∩</sup> <sup>∂</sup>Sg i � � ∪ ∂G<sup>R</sup>

<sup>i</sup> ∩ ∂Hij and ∂G<sup>R</sup>

Since the region Ω is assumed to be static, it holds that μ<sup>i</sup>

<sup>i</sup> are points on a circle with center qi

three disjoint sets as follows

52 Applications of Mobile Robots

<sup>i</sup> <sup>∩</sup> <sup>∂</sup>Sg

and the region, while ∂G<sup>R</sup>

<sup>i</sup> , <sup>∂</sup>G<sup>R</sup>

lines respectively.

<sup>i</sup> <sup>∩</sup> <sup>∂</sup>S<sup>g</sup>

This subset Ω<sup>s</sup>

the disk U<sup>0</sup>

positioning uncertainty ri

Figure 3. Decomposition of ∂GR

of the agent's sensing disk, ∂GR

<sup>i</sup> ∩ ∂Ω, ∂G<sup>R</sup>

2.3.4. Constraining agents inside the region

In order to avoid such a situation, a subset Ω<sup>s</sup>

<sup>i</sup> ð Þ¼ ri f g <sup>q</sup><sup>∈</sup> <sup>Ω</sup> : <sup>∥</sup>q<sup>∥</sup> <sup>≤</sup> ri which is <sup>Ω</sup><sup>s</sup>

where ∂G<sup>R</sup>

∂G<sup>R</sup> <sup>i</sup> <sup>∩</sup> <sup>∂</sup>Sg

q∈∂GR

<sup>i</sup> . The boundary ∂G<sup>R</sup>

j∈ Ni

<sup>i</sup> denotes part of the r-limited cell's boundary that is also part of the boundary

, it holds that μ<sup>i</sup>

<sup>i</sup> is in the general case different among agents due to their differing measures of

. This subset of Ω is computed as the Minkowski difference of Ω with

<sup>i</sup> <sup>¼</sup> <sup>q</sup><sup>∈</sup> <sup>Ω</sup> : <sup>q</sup> <sup>þ</sup> <sup>U</sup><sup>0</sup>

∂G<sup>R</sup> <sup>i</sup> ∩ ∂Hij � � " #

<sup>i</sup> ∩ ∂Ω denotes the common boundary between the r-limited cell

<sup>j</sup> ∩ ∂Hji are shown in solid green, red, blue and purple

<sup>i</sup> <sup>¼</sup> <sup>I</sup>2, <sup>∀</sup>q∈∂GR

<sup>i</sup> ⊆ Ω is used instead, instead of the region Ω.

<sup>i</sup> <sup>⊆</sup> <sup>Ω</sup> � �, i <sup>∈</sup>In.

<sup>i</sup> <sup>¼</sup> <sup>0</sup>, <sup>∀</sup>q∈∂G<sup>R</sup>

<sup>i</sup> ∩ ∂Hij denotes parts of the boundary that consist of hyperbolic arcs

<sup>i</sup> <sup>∩</sup> <sup>∂</sup><sup>Ω</sup> � � <sup>∪</sup> <sup>⋃</sup>

induced by some neighboring agent j. This decomposition is presented in Figure 3 where

grouping the hyperbolic arcs in pairs and multiplying by <sup>α</sup><sup>i</sup> we get (15). □

When the control law (15) is used, there can be cases where the positioning uncertainty regions of some agent does not remain entirely inside Ω, i.e. it is possible that Ui ∩ Ω 6¼ Ui for some agent i. This has the implication that the control law (15) may lead some agent i outside the region Ω, given the fact that an agent may reside anywhere within its positioning uncertainty region Ui.

<sup>i</sup> into disjoint sets and corresponding normal vectors.

is only affected by the movement of agent i at the induced hyperbolic arc ∂G<sup>R</sup>

<sup>i</sup> can be decomposed into

, (16)

<sup>i</sup> ∩ ∂Ω. In addition, since

<sup>i</sup> . Finally, GR

<sup>j</sup> ∩ ∂Hji and by

j

<sup>i</sup> <sup>∩</sup> <sup>∂</sup>Sg

An indicative simulation is presented in this section. This simulation is identical to the one presented in Section 2.2.4 with the only difference being the addition of positioning uncertainty to the agents.

The initial and final agent configurations are shown in Figure 4a and c respectively where the agent positioning uncertainty regions are shown as black circles, the boundaries of their sensing disks are shown as dashed black lines, the boundaries of their cells are marked by solid black lines while their interiors are filled in color. The agent trajectories are shown in Figure 4b with the initial positions marked by dots and the final positions by circles. It is observed that the agents successfully deploy inside the region, increasing the covered area in the process. In order to provide a more objective measure of the agents' performance, the two metrics described in Section 2.2.4 are used which in the case of uncertainty positioning are more formally defined as

$$\mathcal{H}^{\ell^\*} = 100 \frac{\mathcal{H}}{\int\_{\Omega} \phi(q) \mathbf{d}q}, \qquad \mathcal{H}^d = 100 \frac{\mathcal{H}}{\sum\_{i \in I\_n} \int\_{S\_i^\ell} \mathbf{d}q}.$$

Figure 4d shows H <sup>a</sup> in solid blue and H <sup>r</sup> in dashed red with their final values being 94.0 and 70.0% respectively. In this simulation we observe that although H <sup>a</sup> reaches a high value, this is

Figure 4. Simulation study II: (a) initial configuration, (b) agent trajectories, (c) final configuration and (d) evolution of the coverage objective over time.

not the case with H <sup>r</sup> . The first reason for this result is the fact that the computation of H is based on the agents' guaranteed sensing patterns Sg <sup>i</sup> , which by definition are lower in area than their respective sensing patterns Si. Moreover, due to the definition of H being conservative, only the area of the r-limited cells GR <sup>i</sup> counts toward the value of H , thus parts of the neutral region O that are covered by the agents do not contribute to H . Consequently, in the case of the AWGV partitioning (13), coverage objective (14) and control law (15), it is expected for H <sup>r</sup> to achieve a lower value.

that qi <sup>∈</sup> Int Sb

i

be effectively utilized.

2.4.2. Space partitioning

sensed region ∪

cell Wi as follows

can be defined as

being left unassigned.

i∈ In

and that its boundary ∂Sb

radius Ri of a base sensing pattern be defined as Ri S<sup>b</sup>

<sup>i</sup> can be described by a set of parametric equations. Let the

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

∥q∥. This is the maximum distance

http://dx.doi.org/10.5772/intechopen.78940

55

<sup>i</sup> : (20)

i <sup>¼</sup> max q∈∂Sb i

from the origin, which is also the base sensing pattern's center of rotation, to its boundary.

Si qi ; θ<sup>i</sup>

The sensing pattern of agent i as a function of its position qi and orientation θi, can be computed by rotating around the origin and translating its base sensing pattern as follows

<sup>¼</sup> qi <sup>þ</sup> <sup>R</sup>ð Þ <sup>θ</sup><sup>i</sup> Sb

By allowing a different base sensing pattern for each agent, teams of heterogeneous agents can

Since the goal of the mobile agent team is the maximization of the covered area using their sensors, while also taking into account the space density function, we define the coverage objective as in (3). The control objective is the design of a distributed control law for the mobile agents in order to guarantee monotonic increase of the coverage objective H over time.

The first step in designing a distributed control law is finding a method to distribute the computation of the coverage objective H . Due to the agents' anisotropic sensing patterns, the partitioning scheme employed in this case is highly different from Voronoi-like partitioning schemes. Instead of partitioning the whole region Ω based on the agents' positions, only the

the part of Ω that only itself is able to sense, with parts being sensed by multiple or no agents

Given a planar region Ω and a set of sensing patterns S ¼ f g S1; …; Sn , each agent i is assigned a

The part of Ω sensed by multiple agents is left unassigned but still contributes toward the coverage objective H . This part is called the common region and is computed as follows

We can define the neighbors of agent i as those agents that affect the computation of its cell. Since the computation of the cells relies entirely on the agents' sensing patterns, the neighbors

j ∈In∖i

i∈ In

Wið Þ¼ Ω; S ð Þ Ω ∩ Si ∖ ⋃

Wcð Þ¼ Ω; S Ω ∩ ⋃

Having defined the cells and the common region, it holds that ⋃

Si is partitioned based on the agents' sensing patterns. Each agent is assigned

Sj, i∈ In:

i∈ In

Ni <sup>¼</sup> <sup>j</sup>∈In∖<sup>i</sup> : Si <sup>∩</sup> Sj 6¼ ; : (22)

ð Þ Si∖Wi : (21)

Wi ∪ Wc ⊆ Ω .

Si ¼ ⋃ i ∈In

#### 2.4. Heterogeneous agents with anisotropic sensing

Although the omnidirectional sensors examined in the previous two sections can significantly simplify the problem formulation and solution, they are usually inadequate for precise modeling of real-life sensors. For this reason there have been several differing approaches to area coverage using agents with anisotropic sensing patterns [37–40]. In this section we will follow the methodology presented in [41] which is a distributed optimization technique resulting in a gradient-based control law.

#### 2.4.1. Problem statement

A team of n mobile ground agents is deployed inside the region of interest Ω. Given the anisotropic nature of the sensing patterns examined in this section, the mobile agents should be able to change their orientation as well as move around inside the region of interest. A realistic model for a mobile agent with the ability to rotate is that of the differential drive robot whose kinematic model is

$$\begin{aligned} \dot{q}\_i &= \begin{bmatrix} \cos \theta\_i \\ \sin \theta\_i \end{bmatrix} \frac{\rho\_i}{2} \text{ (}\Omega\_i^R + \Omega\_i^L\text{)}, \quad q\_i \in \Omega\_r\\ \dot{\theta}\_i &= \frac{\rho\_i}{l\_i} \text{ (}\Omega\_i^R - \Omega\_i^L\text{)}, \quad \theta\_i \in [-\pi, \ \pi] \end{aligned}$$

where Ω<sup>R</sup> <sup>i</sup> , Ω<sup>L</sup> <sup>i</sup> are the rotational velocities of the right and left wheels, respectively, r<sup>i</sup> is the wheel radius, and li is the length of the wheel axis. In this chapter however a simpler single integrator kinematic model is used for the agents. Each agent i ∈In is approximated by a point mass located at qi ∈ Ω with orientation θ<sup>i</sup> which is governed by the kinematic model described by

$$
\dot{q}\_i = u\_i, \ q \in \Omega, \ u\_i \in \mathbb{R}^2,\tag{18}
$$

$$
\dot{\theta}\_i = \omega\_{\dot{\nu}} \quad \theta\_{\prime} \omega\_i \in \mathbb{R}\_{\prime} \tag{19}
$$

where ω<sup>i</sup> is the rotational velocity control input of the agent. This single integrator model simplifies the derivation of the control law, although the control law can be extended for differential drive robots as well.

We define the base sensing pattern Sb <sup>i</sup> of agent i as the region sensed by the agent when qi <sup>¼</sup> ½ � <sup>0</sup>; <sup>0</sup> <sup>T</sup> and <sup>θ</sup><sup>i</sup> <sup>¼</sup> 0. The only requirements with regards to the base sensing pattern are that qi <sup>∈</sup> Int Sb i and that its boundary ∂Sb <sup>i</sup> can be described by a set of parametric equations. Let the radius Ri of a base sensing pattern be defined as Ri S<sup>b</sup> i <sup>¼</sup> max q∈∂Sb i ∥q∥. This is the maximum distance

from the origin, which is also the base sensing pattern's center of rotation, to its boundary.

The sensing pattern of agent i as a function of its position qi and orientation θi, can be computed by rotating around the origin and translating its base sensing pattern as follows

$$\mathcal{S}\_i(\boldsymbol{q}\_i, \boldsymbol{\Theta}\_i) = \boldsymbol{q}\_i + \mathbf{R}(\boldsymbol{\Theta}\_i) \; \mathcal{S}\_i^b. \tag{20}$$

By allowing a different base sensing pattern for each agent, teams of heterogeneous agents can be effectively utilized.

Since the goal of the mobile agent team is the maximization of the covered area using their sensors, while also taking into account the space density function, we define the coverage objective as in (3). The control objective is the design of a distributed control law for the mobile agents in order to guarantee monotonic increase of the coverage objective H over time.

#### 2.4.2. Space partitioning

not the case with H <sup>r</sup>

54 Applications of Mobile Robots

to achieve a lower value.

gradient-based control law.

2.4.1. Problem statement

whose kinematic model is

<sup>i</sup> , Ω<sup>L</sup>

differential drive robots as well.

We define the base sensing pattern Sb

where Ω<sup>R</sup>

based on the agents' guaranteed sensing patterns Sg

2.4. Heterogeneous agents with anisotropic sensing

q\_

<sup>θ</sup>\_ <sup>i</sup> <sup>¼</sup> <sup>r</sup><sup>i</sup> li

<sup>i</sup> <sup>¼</sup> cos <sup>θ</sup><sup>i</sup> sin θ<sup>i</sup>

" #

Ω<sup>R</sup> <sup>i</sup> � <sup>Ω</sup><sup>L</sup> i � �, <sup>θ</sup><sup>i</sup> <sup>∈</sup> ½ � �π; <sup>π</sup> ,

θ\_

only the area of the r-limited cells GR

. The first reason for this result is the fact that the computation of H is

their respective sensing patterns Si. Moreover, due to the definition of H being conservative,

region O that are covered by the agents do not contribute to H . Consequently, in the case of the AWGV partitioning (13), coverage objective (14) and control law (15), it is expected for H <sup>r</sup>

Although the omnidirectional sensors examined in the previous two sections can significantly simplify the problem formulation and solution, they are usually inadequate for precise modeling of real-life sensors. For this reason there have been several differing approaches to area coverage using agents with anisotropic sensing patterns [37–40]. In this section we will follow the methodology presented in [41] which is a distributed optimization technique resulting in a

A team of n mobile ground agents is deployed inside the region of interest Ω. Given the anisotropic nature of the sensing patterns examined in this section, the mobile agents should be able to change their orientation as well as move around inside the region of interest. A realistic model for a mobile agent with the ability to rotate is that of the differential drive robot

> ri <sup>2</sup> <sup>Ω</sup><sup>R</sup>

wheel radius, and li is the length of the wheel axis. In this chapter however a simpler single integrator kinematic model is used for the agents. Each agent i ∈In is approximated by a point mass located at qi ∈ Ω with orientation θ<sup>i</sup> which is governed by the kinematic model described by

<sup>q</sup>\_<sup>i</sup> <sup>¼</sup> ui, q<sup>∈</sup> <sup>Ω</sup>, ui <sup>∈</sup> <sup>R</sup><sup>2</sup>

where ω<sup>i</sup> is the rotational velocity control input of the agent. This single integrator model simplifies the derivation of the control law, although the control law can be extended for

qi <sup>¼</sup> ½ � <sup>0</sup>; <sup>0</sup> <sup>T</sup> and <sup>θ</sup><sup>i</sup> <sup>¼</sup> 0. The only requirements with regards to the base sensing pattern are

<sup>i</sup> <sup>þ</sup> <sup>Ω</sup><sup>L</sup> i � �, qi <sup>∈</sup> <sup>Ω</sup>,

<sup>i</sup> are the rotational velocities of the right and left wheels, respectively, r<sup>i</sup> is the

<sup>i</sup> , which by definition are lower in area than

, (18)

<sup>i</sup> ¼ ωi, θ, ω<sup>i</sup> ∈ R, (19)

<sup>i</sup> of agent i as the region sensed by the agent when

<sup>i</sup> counts toward the value of H , thus parts of the neutral

The first step in designing a distributed control law is finding a method to distribute the computation of the coverage objective H . Due to the agents' anisotropic sensing patterns, the partitioning scheme employed in this case is highly different from Voronoi-like partitioning schemes. Instead of partitioning the whole region Ω based on the agents' positions, only the sensed region ∪ i∈ In Si is partitioned based on the agents' sensing patterns. Each agent is assigned

the part of Ω that only itself is able to sense, with parts being sensed by multiple or no agents being left unassigned.

Given a planar region Ω and a set of sensing patterns S ¼ f g S1; …; Sn , each agent i is assigned a cell Wi as follows

$$W\_i(\Omega, \mathcal{S}) = (\Omega \cap \mathcal{S}\_i) \backslash \bigcup\_{j \in I\_n \backslash i} \mathcal{S}\_{j\prime} \qquad i \in \mathcal{I}n.$$

The part of Ω sensed by multiple agents is left unassigned but still contributes toward the coverage objective H . This part is called the common region and is computed as follows

$$\mathcal{W}\_{\varepsilon}(\Omega, \mathcal{S}) = \Omega \cap \bigcup\_{i \in I\_n} (\mathcal{S}\_i \backslash \mathcal{W}\_i). \tag{21}$$

Having defined the cells and the common region, it holds that ⋃ i∈ In Si ¼ ⋃ i ∈In Wi ∪ Wc ⊆ Ω .

We can define the neighbors of agent i as those agents that affect the computation of its cell. Since the computation of the cells relies entirely on the agents' sensing patterns, the neighbors can be defined as

$$N\_{\mathbf{i}} = \left\{ \mathbf{j} \in I\_{\mathbf{n}} \, \middle| \, \mathbf{i} : \, \mathbf{S}\_{\mathbf{i}} \cap \mathbf{S}\_{\mathbf{j}} \neq \emptyset \right\}. \tag{22}$$

Moreover, if the maximum base sensing radius <sup>R</sup>max <sup>¼</sup> max i∈ In Ri is known by all agents, and if each agent is able to communicate with all others within a radius

$$\mathbf{C}\_{i} = \mathbf{R}\_{i} + \mathbf{R}^{\text{max}} \,\mathrm{},\tag{23}$$

∂H <sup>∂</sup><sup>t</sup> <sup>¼</sup> <sup>X</sup> i∈In

By selecting the control law

The partial derivative <sup>∂</sup><sup>H</sup>

of agent i and since <sup>∂</sup>ϕð Þ<sup>q</sup>

becomes

where υ<sup>i</sup>

∂H ∂qi

∂qi ∂t þ

ui ¼ αi,u

∂qi

¼ ∂ ∂qi ð

∂qi

<sup>j</sup> is the Jacobian matrix

in solid green, red and blue lines respectively.

Since the region Ω is assumed to be static, it holds that υ<sup>i</sup>

<sup>i</sup> <sup>¼</sup> <sup>υ</sup><sup>c</sup>

three disjoint sets as follows

Eq. (20) we get that υ<sup>i</sup>

∂Wj ∩ ∂Wc, j ∈In it holds that υ<sup>j</sup>

∂H ∂qi ¼ ð

∂Wi

υi

υi <sup>j</sup> <sup>¼</sup> <sup>∂</sup><sup>q</sup> ∂qi

Wi

∂H ∂qi

∂H ∂θ<sup>i</sup>

∂H ∂qi

we can guarantee monotonic increase of the coverage objective.

∂θ<sup>i</sup> <sup>∂</sup><sup>t</sup> <sup>¼</sup> <sup>X</sup> i ∈In

is evaluated as follows

<sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup><sup>q</sup> <sup>þ</sup><sup>X</sup>

, ω<sup>i</sup> ¼ αi,<sup>ω</sup>

j6¼i

∂ ∂qi ð

Wj

Due to the partitioning scheme (24) only the common region Wc is affected by the movement

<sup>i</sup> ni ϕð Þq dq þ

ϕð Þq dq þ

ð

υi

∂Wi ¼ ð Þ ∂Wi ∩ ∂Si ∪ ð Þ ∂Wi ∩ ∂Ω ∪ ð Þ ∂Wi ∩ ∂Wc , (28)

<sup>i</sup> ¼ I2, ∀q∈ ∂Wi ∩ ∂Si. Finally, on all the common boundaries

<sup>i</sup> and nj ¼ �nc, as shown in Figure 5, leaving only an

∂Wc

, q∈ ∂Wj

and ni is the outward unit normal vector on ∂Wi. The boundary ∂Wi can be decomposed into

where ∂Wi ∩ ∂Si denotes part of the cell's boundary that is also part of the boundary of the agent's sensing disk, ∂Wi ∩ ∂Ω denotes the common boundary between the cell and the region, while ∂Wi ∩ ∂Wc denotes the common boundary of the cell and the common region. This decomposition is presented in Figure 5 where ∂Wi ∩ ∂Si, ∂Wi ∩ ∂Ω and ∂Wi ∩ ∂Wc are shown

integral over ∂Wi ∩ ∂Si. By multiplying with αi,u we get (26). The same procedure is used for the derivation of the rotational part of the control law (27). □

¼ 0, by using the Leibniz integral rule [32], the previous equation

<sup>c</sup> ncϕð Þq dq

∂H ∂qi q\_<sup>i</sup> þ ∂H ∂θ<sup>i</sup> θ\_ <sup>i</sup> <sup>¼</sup> <sup>X</sup> i∈ In

> ∂H ∂θ<sup>i</sup>

∂H ∂qi ui þ ∂H ∂θ<sup>i</sup> ωi:

http://dx.doi.org/10.5772/intechopen.78940

57

, αi,u, αi,<sup>ω</sup> > 0,

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

∂ ∂qi ð

Wc

ϕð Þq dq:

<sup>i</sup> ¼ 0, ∀q∈ ∂Wi ∩ ∂Ω. In addition, from

then it is guaranteed it will be able to communicate with all of its neighbors Ni. By using the neighbor definition, the proposed partitioning scheme can be computed in a distributed manner as follows

$$W\_i(\Omega, \mathcal{S}) = (\Omega \cap \mathcal{S}\_i) \backslash \bigcup\_{j \in N\_i \backslash i} \mathcal{S}\_{j} \quad i \in I\_n. \tag{24}$$

Remark 2.2. The partitioning scheme (24) may result in the cell of some agent being empty or consisting of multiple disjoint regions. It should be noted however that such cases are handled successfully by the control law presented in Section 2.4.3.

Thus by utilizing the aforementioned partitioning scheme, the coverage objective H can be computed as follows

$$\mathcal{H}' = \sum\_{i \in I\_n} \int\_{W\_i} \phi(q) \, \, dq + \int\_{W\_c} \phi(q) \, \, dq. \tag{25}$$

Since H can be written as a sum of integrals over the assigned cells and since an agent can compute its own cell using information just from its neighbors, the computation of H is distributed.

#### 2.4.3. Control law formulation

Having found a partitioning scheme that allows distributed computation of the coverage objective H , what is left is the derivation of a distributed control law for its maximization.

Theorem 2.3. For a team of mobile ground agents with kinematics (18, 19), sensing performance (20) and using the partitioning (24), the control scheme

$$u\_i = \alpha\_{i,u} \int\_{\substack{\partial \mathcal{W}\_i \cap \partial \mathcal{S}\_i}} n\_i \, \phi(q) \, \, \mathbf{d}q,\tag{26}$$

$$
\omega\_{i} = \alpha\_{i,\omega} \int\_{\partial \mathcal{W}\_{i} \cap \partial \mathcal{S}\_{i}} n\_{i} \, \operatorname{\mathbf{R}} \left( \frac{\pi}{2} \right) \left( q - q\_{i} \right) \, \phi(q) \, \operatorname{d}q \,\tag{27}
$$

where αi,u, αi,<sup>ω</sup> are positive constants and ni is the outward unit normal vector on ∂Wi, leading the agent to trajectories that result in monotonic increase of the coverage objective (25).

Proof. We start by evaluating the time derivative of the objective using the chain rule and the agent dynamics (18, 19)

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents http://dx.doi.org/10.5772/intechopen.78940 57

$$\frac{\partial \mathcal{M}^{\boldsymbol{\theta}}}{\partial t} = \sum\_{i \in I\_{\boldsymbol{u}}} \frac{\partial \mathcal{M}^{\boldsymbol{\theta}}}{\partial q\_{i}} \frac{\partial q\_{i}}{\partial t} + \frac{\partial \mathcal{M}^{\boldsymbol{\theta}}}{\partial \boldsymbol{\theta}\_{i}} \frac{\partial \boldsymbol{\theta}\_{i}}{\partial t} = \sum\_{i \in I\_{\boldsymbol{u}}} \frac{\partial \mathcal{M}^{\boldsymbol{\theta}}}{\partial q\_{i}} \dot{q}\_{i} + \frac{\partial \mathcal{M}^{\boldsymbol{\theta}}}{\partial \boldsymbol{\theta}\_{i}} \dot{\boldsymbol{\theta}}\_{i} = \sum\_{i \in I\_{\boldsymbol{u}}} \frac{\partial \mathcal{M}^{\boldsymbol{\theta}}}{\partial q\_{i}} \boldsymbol{u}\_{i} + \frac{\partial \mathcal{M}}{\partial \boldsymbol{\theta}\_{i}} \boldsymbol{\omega}\_{i}.$$

By selecting the control law

Moreover, if the maximum base sensing radius <sup>R</sup>max <sup>¼</sup> max

fully by the control law presented in Section 2.4.3.

and using the partitioning (24), the control scheme

manner as follows

56 Applications of Mobile Robots

computed as follows

2.4.3. Control law formulation

agent dynamics (18, 19)

distributed.

each agent is able to communicate with all others within a radius

<sup>ℋ</sup> <sup>¼</sup> <sup>X</sup> i∈ In

i∈ In

then it is guaranteed it will be able to communicate with all of its neighbors Ni. By using the neighbor definition, the proposed partitioning scheme can be computed in a distributed

Remark 2.2. The partitioning scheme (24) may result in the cell of some agent being empty or consisting of multiple disjoint regions. It should be noted however that such cases are handled success-

Thus by utilizing the aforementioned partitioning scheme, the coverage objective H can be

ϕð Þq dq þ

Since H can be written as a sum of integrals over the assigned cells and since an agent can compute its own cell using information just from its neighbors, the computation of H is

Having found a partitioning scheme that allows distributed computation of the coverage objective H , what is left is the derivation of a distributed control law for its maximization.

Theorem 2.3. For a team of mobile ground agents with kinematics (18, 19), sensing performance (20)

ð

∂Wi ∩ ∂Si

ni <sup>R</sup> <sup>π</sup> 2 � �

where αi,u, αi,<sup>ω</sup> are positive constants and ni is the outward unit normal vector on ∂Wi, leading the

Proof. We start by evaluating the time derivative of the objective using the chain rule and the

q � qi

ui ¼ αi,u

ð

∂Wi ∩ ∂Si

agent to trajectories that result in monotonic increase of the coverage objective (25).

ω<sup>i</sup> ¼ αi,<sup>ω</sup>

ð

Wc

j∈ Ni∖i

Wið Þ¼ Ω; S ð Þ Ω ∩ Si ∖ ⋃

ð

Wi

Ci <sup>¼</sup> Ri <sup>þ</sup> <sup>R</sup>max, (23)

Ri is known by all agents, and if

Sj, i ∈In: (24)

ϕð Þq dq: (25)

ni ϕð Þq dq, (26)

� � <sup>ϕ</sup>ð Þ<sup>q</sup> <sup>d</sup>q, (27)

$$
\mu\_i = \alpha\_{i,u} \frac{\partial \mathcal{H}}{\partial q\_i}, \qquad \omega\_i = \alpha\_{i,u} \frac{\partial \mathcal{H}}{\partial \Theta\_i}, \qquad \alpha\_{i,u} \alpha\_{i,u} > 0,
$$

we can guarantee monotonic increase of the coverage objective.

The partial derivative <sup>∂</sup><sup>H</sup> ∂qi is evaluated as follows

$$\frac{\partial \mathcal{M}}{\partial q\_i} = \frac{\partial}{\partial q\_i} \int\_{W\_i} \phi(q) \, \, \mathrm{d}q \, \, \, + \sum\_{j \neq i} \frac{\partial}{\partial q\_i} \int\_{W\_j} \phi(q) \, \, \, \mathrm{d}q \, \, \, \, \, \, \, \, \, \frac{\partial}{\partial q\_i} \int\_{W\_\varepsilon} \phi(q) \, \, \, \mathrm{d}q.$$

Due to the partitioning scheme (24) only the common region Wc is affected by the movement of agent i and since <sup>∂</sup>ϕð Þ<sup>q</sup> ∂qi ¼ 0, by using the Leibniz integral rule [32], the previous equation becomes

$$\frac{\partial \mathcal{H}}{\partial q\_i} = \int\_{\partial V\_i} \nu\_i^{\dot{e}} \ n\_i \, \phi(q) \, \, \mathbf{d}q \, \, \, + \, \int\_{\partial V\_c} \nu\_c^{\dot{e}} \ n\_c \phi(q) \, \, \mathbf{d}q$$

where υ<sup>i</sup> <sup>j</sup> is the Jacobian matrix

$$\nu\_j^i = \frac{\partial q}{\partial q\_i}, \quad q \in \partial W\_j$$

and ni is the outward unit normal vector on ∂Wi. The boundary ∂Wi can be decomposed into three disjoint sets as follows

$$
\partial \mathcal{W}\_i = (\partial \mathcal{W}\_i \cap \partial \mathcal{S}\_i) \cup (\partial \mathcal{W}\_i \cap \partial \mathcal{Q}) \cup (\partial \mathcal{W}\_i \cap \partial \mathcal{W}\_c), \tag{28}
$$

where ∂Wi ∩ ∂Si denotes part of the cell's boundary that is also part of the boundary of the agent's sensing disk, ∂Wi ∩ ∂Ω denotes the common boundary between the cell and the region, while ∂Wi ∩ ∂Wc denotes the common boundary of the cell and the common region. This decomposition is presented in Figure 5 where ∂Wi ∩ ∂Si, ∂Wi ∩ ∂Ω and ∂Wi ∩ ∂Wc are shown in solid green, red and blue lines respectively.

Since the region Ω is assumed to be static, it holds that υ<sup>i</sup> <sup>i</sup> ¼ 0, ∀q∈ ∂Wi ∩ ∂Ω. In addition, from Eq. (20) we get that υ<sup>i</sup> <sup>i</sup> ¼ I2, ∀q∈ ∂Wi ∩ ∂Si. Finally, on all the common boundaries ∂Wj ∩ ∂Wc, j ∈In it holds that υ<sup>j</sup> <sup>i</sup> <sup>¼</sup> <sup>υ</sup><sup>c</sup> <sup>i</sup> and nj ¼ �nc, as shown in Figure 5, leaving only an integral over ∂Wi ∩ ∂Si. By multiplying with αi,u we get (26). The same procedure is used for the derivation of the rotational part of the control law (27). □

Figure 5. Decomposition of ∂Wi into disjoint sets and corresponding normal vectors.

#### 2.4.4. Simulation study III

An indicative simulation is presented in this section. The region Ω, the space density function ϕð Þq and the agent initial positions are the same as in the simulation presented in Section 2.2.4. In this simulation however the agents are equipped with heterogeneous sensors with elliptical sensing patterns.

Simulation Study III was repeated by approximating the agents' elliptical sensing patterns with their maximal inscribed circles. The initial agent configuration, agent trajectories and final agent configuration are shown in Figure 7a, b and c respectively. It is observed that the agent's performance is decreased significantly when using this underapproximation of their sensing patterns. In order to provide a more objective measure of the agents' performance, the two metrics defined in Eq. (8) are used. Figure 7d shows H <sup>a</sup> in solid blue and H <sup>r</sup> in dashed red with their final values being 100% and 35.2% respectively, indicating a 62.4% decrease in the

Figure 7. Simulation study IV: (a) initial configuration, (b) agent trajectories, (c) final configuration and (d) evolution of

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

http://dx.doi.org/10.5772/intechopen.78940

59

An experimental implementation of a simplified version of one of the previously examined control schemes is briefly presented in this section. This experimental study first appeared and is presented in greater detail in [42]. The experiment consisted of three differential-drive robots, a visual pose tracking system using fiducial markers and a computer communicating with the robots and pose tracking system via a WiFi router. The methodology presented in Section 2.3 was used in order to take into account the positioning uncertainty of the pose tracking system. The experimental results are compared with a simulation using the same

The robots used in the experiment were the differential-drive AmigoBots by Omron Adept MobileRobots. The robots are 33 cm � 28 cm � 15 cm in size, weigh 3:6 kg and are able to carry a payload of up to 1 kg. Their maximum linear and rotational velocities are <sup>v</sup>max <sup>¼</sup> 1 m=s and <sup>ω</sup>max <sup>¼</sup> 100o=s. Although these robots are equipped with encoders measuring 39; 000 ticks<sup>=</sup> revolution which can be used for estimating their pose, an external pose tracking system was used instead due to the encoders' drifting error. Since the AmigoBots lack any omnidirectional sensors, for the sake of the control law it was assumed that they were equipped with sensors

The external pose tracking system consists of a USB camera and an ODROID-XU4 computer. Pose tracking is achieved by attaching a fiducial marker on top of each robot and using the

coverage of Ω compared to Simulation Study III.

2.5. Experimental implementation

the coverage objective over time.

initial conditions.

2.5.1. Experimental setup

with a common sensing radius of R ¼ 0:3 m.

The initial and final agent configurations are shown in Figure 6a and c respectively where the agent positions are marked by black dots, the agent orientations are marked by black arrows, the boundaries of their sensing disks are shown as dashed black lines, the boundaries of their cells are marked by solid black lines while their interiors are filled in color. The agent trajectories are shown in Figure 6b with the initial positions marked by dots and the final positions by circles. It is observed that the agents successfully deploy inside the region, increasing the covered area in the process. In order to provide a more objective measure of the agents' performance, the two metrics defined in Eq. (8) are used. Figure 6d shows H <sup>a</sup> in solid blue and H <sup>r</sup> in dashed red with their final values being 91.3 and 93.5% respectively. This indicates that the final configuration results in both high coverage of Ω and efficient use of the agents sensors.

#### 2.4.5. Simulation study IV

This simulation study serves to highlight the need for taking into account the agents' anisotropic sensing patterns instead of approximating them with circular ones. To that end,

Figure 6. Simulation study III: (a) initial configuration, (b) agent trajectories, (c) final configuration and (d) evolution of the coverage objective over time.

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents http://dx.doi.org/10.5772/intechopen.78940 59

Figure 7. Simulation study IV: (a) initial configuration, (b) agent trajectories, (c) final configuration and (d) evolution of the coverage objective over time.

Simulation Study III was repeated by approximating the agents' elliptical sensing patterns with their maximal inscribed circles. The initial agent configuration, agent trajectories and final agent configuration are shown in Figure 7a, b and c respectively. It is observed that the agent's performance is decreased significantly when using this underapproximation of their sensing patterns. In order to provide a more objective measure of the agents' performance, the two metrics defined in Eq. (8) are used. Figure 7d shows H <sup>a</sup> in solid blue and H <sup>r</sup> in dashed red with their final values being 100% and 35.2% respectively, indicating a 62.4% decrease in the coverage of Ω compared to Simulation Study III.

#### 2.5. Experimental implementation

2.4.4. Simulation study III

58 Applications of Mobile Robots

2.4.5. Simulation study IV

the coverage objective over time.

patterns.

An indicative simulation is presented in this section. The region Ω, the space density function ϕð Þq and the agent initial positions are the same as in the simulation presented in Section 2.2.4. In this simulation however the agents are equipped with heterogeneous sensors with elliptical sensing

The initial and final agent configurations are shown in Figure 6a and c respectively where the agent positions are marked by black dots, the agent orientations are marked by black arrows, the boundaries of their sensing disks are shown as dashed black lines, the boundaries of their cells are marked by solid black lines while their interiors are filled in color. The agent trajectories are shown in Figure 6b with the initial positions marked by dots and the final positions by circles. It is observed that the agents successfully deploy inside the region, increasing the covered area in the process. In order to provide a more objective measure of the agents' performance, the two metrics defined in Eq. (8) are used. Figure 6d shows H <sup>a</sup> in solid blue and H <sup>r</sup> in dashed red with their final values being 91.3 and 93.5% respectively. This indicates that the final configuration

This simulation study serves to highlight the need for taking into account the agents' anisotropic sensing patterns instead of approximating them with circular ones. To that end,

Figure 6. Simulation study III: (a) initial configuration, (b) agent trajectories, (c) final configuration and (d) evolution of

results in both high coverage of Ω and efficient use of the agents sensors.

Figure 5. Decomposition of ∂Wi into disjoint sets and corresponding normal vectors.

An experimental implementation of a simplified version of one of the previously examined control schemes is briefly presented in this section. This experimental study first appeared and is presented in greater detail in [42]. The experiment consisted of three differential-drive robots, a visual pose tracking system using fiducial markers and a computer communicating with the robots and pose tracking system via a WiFi router. The methodology presented in Section 2.3 was used in order to take into account the positioning uncertainty of the pose tracking system. The experimental results are compared with a simulation using the same initial conditions.

#### 2.5.1. Experimental setup

The robots used in the experiment were the differential-drive AmigoBots by Omron Adept MobileRobots. The robots are 33 cm � 28 cm � 15 cm in size, weigh 3:6 kg and are able to carry a payload of up to 1 kg. Their maximum linear and rotational velocities are <sup>v</sup>max <sup>¼</sup> 1 m=s and <sup>ω</sup>max <sup>¼</sup> 100o=s. Although these robots are equipped with encoders measuring 39; 000 ticks<sup>=</sup> revolution which can be used for estimating their pose, an external pose tracking system was used instead due to the encoders' drifting error. Since the AmigoBots lack any omnidirectional sensors, for the sake of the control law it was assumed that they were equipped with sensors with a common sensing radius of R ¼ 0:3 m.

The external pose tracking system consists of a USB camera and an ODROID-XU4 computer. Pose tracking is achieved by attaching a fiducial marker on top of each robot and using the ArUco [43] library to estimate the pose of these markers. As is the case with all positioning systems, ArUco has a certain degree of uncertainty in its pose estimations. In order to get an estimate of this uncertainty, a fiducial marked was placed on the vertices and the centroid of the region Ω resulting in a maximum error of 0:032 m, which was used as the measure of positioning uncertainty r for all robots.

The control scheme was implemented as a loop in the main computer with an iteration period of Ts ¼ 0:1 seconds. At each iteration, a simplified version of the control law (15) is computed for each agent, and from that, a target point q<sup>t</sup> <sup>i</sup> is derived for each agent. Then a feedback controller is used in order to lead each robot to each respective target point. Once all robots are within a predefined distance dt <sup>¼</sup> <sup>0</sup>:02 m of their respective target points, new target points are computed from the robots' current positions. The feedback control law used for each robot was

$$\upsilon\_i = \min\left(\frac{\|q\_i^t - q\_i\|}{T\_s}, \sigma^{\max}\right) \cos\left(d\theta\_i\right), \quad \omega\_i = \min\left(\frac{|d\theta\_i|}{T\_s}, \omega^{\max}\right) \sin\left(d\theta\_i\right).$$

the experimental trajectories have many turns due to the robots moving to target points. The robots' final positions have an error of 9:27% the diameter of Ω between the experiment and the simulation. This large error is attributed to the difference between the implemented control laws as well as the existence of multiple global optima for this particular coverage setup. Figure shows the evolution of the metric H <sup>a</sup> over time for the experiment in blue and the simulation in red where it is seen that it increased from 83:70 to 98:95% in the experiment. Although in the case of the experiment its increase was not monotonic, this is to be expected as the implemented control law differed from the theoretical one. The lower convergence speed is also attributed to this difference as well as the constraints on the robots' translational and

Figure 9. Experiment: (a) initial configuration, (b) final configuration and (c) evolution of the coverage objective over time

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

http://dx.doi.org/10.5772/intechopen.78940

61

This chapter presented an overview of past and current work on area coverage problems. A strong theoretical background has been provided, along with indicative simulations results and an experimental implementation of one of the presented control schemes. The problem of multiagent area coverage still offers possibilities for original research. One possible extension would be the usage of more realistic sensor models, such as visual sensors. The usage of visual sensors can result in the incorporation of coverage quality metrics in the objective or in variable sensing patterns in the case of pan-tilt-zoom cameras. Another aspect of multirobot area coverage problem that has not been studied thoroughly yet is the development of communication systems and algorithms that allow the agents to exchange information in a distributed manner. Finally, implementations in actual robotic systems in order to solve practical problems

rotational velocities.

for the experiment and simulation.

are not yet common.

Conflict of interest

The authors declare no conflict of interest.

3. Conclusions and future work

where qi and θ<sup>i</sup> are the robot's current position and orientation, vi and ω<sup>i</sup> the robots linear and rotational velocity control inputs respectively and <sup>d</sup>θ<sup>i</sup> <sup>¼</sup> <sup>∡</sup> <sup>q</sup><sup>t</sup> <sup>i</sup> � qi � <sup>θ</sup>i.

#### 2.5.2. Experimental results

The robots' initial configuration, which is common between the experiment and simulation is shown in Figure 8a. The final configurations of the experiment and the simulation are shown in Figure 8c and d, respectively. The boundaries of the agents' positioning uncertainty regions are shown as black circles, the boundaries of their sensing disks are shown in dashed black line and the boundaries of their cells are marked by solid black lines while their interiors are filled in color. Some photographs of the robots' initial and final configurations are presented in Figure 9a and b respectively where the ArUco fiducial markers can be seen. In both the experiment and the simulation it is observed from the robots' final configurations that their guaranteed sensed regions are completely contained within their respective AWGV cells, i.e. Sg <sup>i</sup> ⊂ Gi, ∀i∈ In, which is a globally optimal configuration. The robots' trajectories are depicted in Figure 8b in blue for the experiment and in red for the simulation, with the initial and final positions marked by dots and circles respectively. The simulation trajectories are smooth but

Figure 8. Experiment: (a) initial configuration, (b) experimental (blue) and simulated (red) robot trajectories, (c) experiment final configuration and (d) simulation final configuration.

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents http://dx.doi.org/10.5772/intechopen.78940 61

Figure 9. Experiment: (a) initial configuration, (b) final configuration and (c) evolution of the coverage objective over time for the experiment and simulation.

the experimental trajectories have many turns due to the robots moving to target points. The robots' final positions have an error of 9:27% the diameter of Ω between the experiment and the simulation. This large error is attributed to the difference between the implemented control laws as well as the existence of multiple global optima for this particular coverage setup. Figure shows the evolution of the metric H <sup>a</sup> over time for the experiment in blue and the simulation in red where it is seen that it increased from 83:70 to 98:95% in the experiment. Although in the case of the experiment its increase was not monotonic, this is to be expected as the implemented control law differed from the theoretical one. The lower convergence speed is also attributed to this difference as well as the constraints on the robots' translational and rotational velocities.

#### 3. Conclusions and future work

ArUco [43] library to estimate the pose of these markers. As is the case with all positioning systems, ArUco has a certain degree of uncertainty in its pose estimations. In order to get an estimate of this uncertainty, a fiducial marked was placed on the vertices and the centroid of the region Ω resulting in a maximum error of 0:032 m, which was used as the measure of

The control scheme was implemented as a loop in the main computer with an iteration period of Ts ¼ 0:1 seconds. At each iteration, a simplified version of the control law (15) is computed

controller is used in order to lead each robot to each respective target point. Once all robots are within a predefined distance dt <sup>¼</sup> <sup>0</sup>:02 m of their respective target points, new target points are computed from the robots' current positions. The feedback control law used for each robot was

where qi and θ<sup>i</sup> are the robot's current position and orientation, vi and ω<sup>i</sup> the robots linear and

The robots' initial configuration, which is common between the experiment and simulation is shown in Figure 8a. The final configurations of the experiment and the simulation are shown in Figure 8c and d, respectively. The boundaries of the agents' positioning uncertainty regions are shown as black circles, the boundaries of their sensing disks are shown in dashed black line and the boundaries of their cells are marked by solid black lines while their interiors are filled in color. Some photographs of the robots' initial and final configurations are presented in Figure 9a and b respectively where the ArUco fiducial markers can be seen. In both the experiment and the simulation it is observed from the robots' final configurations that their guaranteed sensed regions are completely contained within their respective AWGV cells, i.e.

<sup>i</sup> ⊂ Gi, ∀i∈ In, which is a globally optimal configuration. The robots' trajectories are depicted in Figure 8b in blue for the experiment and in red for the simulation, with the initial and final positions marked by dots and circles respectively. The simulation trajectories are smooth but

Figure 8. Experiment: (a) initial configuration, (b) experimental (blue) and simulated (red) robot trajectories, (c) experi-

cos ð Þ <sup>d</sup>θ<sup>i</sup> , <sup>ω</sup><sup>i</sup> <sup>¼</sup> min <sup>∣</sup>dθi<sup>∣</sup>

Ts

<sup>i</sup> � qi � <sup>θ</sup>i.

; ωmax 

sin ð Þ dθ<sup>i</sup> ,

<sup>i</sup> is derived for each agent. Then a feedback

positioning uncertainty r for all robots.

60 Applications of Mobile Robots

vi <sup>¼</sup> min <sup>∥</sup>qt

2.5.2. Experimental results

Sg

for each agent, and from that, a target point q<sup>t</sup>

<sup>i</sup> � qi ∥

; vmax 

rotational velocity control inputs respectively and <sup>d</sup>θ<sup>i</sup> <sup>¼</sup> <sup>∡</sup> <sup>q</sup><sup>t</sup>

Ts

ment final configuration and (d) simulation final configuration.

This chapter presented an overview of past and current work on area coverage problems. A strong theoretical background has been provided, along with indicative simulations results and an experimental implementation of one of the presented control schemes. The problem of multiagent area coverage still offers possibilities for original research. One possible extension would be the usage of more realistic sensor models, such as visual sensors. The usage of visual sensors can result in the incorporation of coverage quality metrics in the objective or in variable sensing patterns in the case of pan-tilt-zoom cameras. Another aspect of multirobot area coverage problem that has not been studied thoroughly yet is the development of communication systems and algorithms that allow the agents to exchange information in a distributed manner. Finally, implementations in actual robotic systems in order to solve practical problems are not yet common.

#### Conflict of interest

The authors declare no conflict of interest.

#### Author details

Sotiris Papatheodorou and Anthony Tzes\*

\*Address all correspondence to: anthony.tzes@nyu.edu

New York University Abu Dhabi, Engineering Division, Abu Dhabi, United Arab Emirates

[11] Kantaros Y, Zavlanos M. Distributed communication-aware coverage control by mobile

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

http://dx.doi.org/10.5772/intechopen.78940

63

[12] Bullo F, Carli R, Frasca P. Gossip coverage control for robotic networks: Dynamical systems on the space of partitions. SIAM Journal on Control and Optimization. 2012;50(1):419-447

[13] Breitenmoser A, Schwager M, Metzger JC, Siegwart R, Rus D. Voronoi coverage of nonconvex environments with a group of networked robots. In: Proceedings IEEE International Conference on Robotics and Automation (ICRA). Anchorage, Alaska, USA: IEEE;

[14] Stergiopoulos Y, Thanou M, Tzes A. Distributed collaborative coverage-control schemes for non-convex domains. IEEE Transactions on Automatic Control. 2015;60(9):2422-2427

[15] Alitappeh RJ, Jeddisaravi K, Guimarães FG. Multiobjective multi-robot deployment in a

[16] Franco C, Stipanović DM, López-Nicolás G, Sagüés C, Llorente S. Persistent coverage control for a team of agents with collision avoidance. European Journal of Control. 2015;

[17] Breitenmoser A, Martinoli A. On combining multi-robot coverage and reciprocal collision avoidance. In: Distributed Autonomous Robotic Systems. Tokyo: Springer; 2016. pp. 49-64

[18] Cortés J, Bullo F. Coordination and geometric optimization via distributed dynamical

[19] Nguyen MT, Rodrigues L, Maniu CS, Olaru S. Discretized optimal control approach for dynamic multi-agent decentralized coverage. In: Proceedings IEEE International Sympo-

[20] Nowzari C, Cortés J. Self-triggered coordination of robotic networks for optimal deploy-

[21] Mavrommati A, Tzorakoleftherakis E, Abraham I, Murphey TD. Real-time area coverage and target localization using receding-horizon ergodic exploration. IEEE Transactions on

[22] Bentz W, Panagou D. 3D dynamic coverage and avoidance control in power-constrained UAV surveillance networks. In: Unmanned Aircraft Systems (ICUAS), 2017 International

[23] Tzes M, Papatheodorou S, Tzes A. Visual area coverage by heterogeneous aerial agents under imprecise localization. IEEE Control Systems Letters. Oct 2018;2(4):623-628. ISSN:

[24] Schwager M, Julian BJ, Angermann M, Rus D. Eyes in the sky: Decentralized control for the deployment of robotic camera networks. Proceedings of the IEEE. 2011;99(9):1541-1561 [25] Papatheodorou S, Tzes A, Stergiopoulos Y. Collaborative visual area coverage. Robotics

and Autonomous Systems. June 2017;92:126-138. ISSN: 0921-8890

sium on Intelligent Control (ISIC), IEEE. Zadar, Croatia; September 2016. pp. 1-6

systems. SIAM Journal on Control and Optimization. 2005;44(5):1543-1574

sensor networks. Automatica. 2016;63:209-220

dynamic environment. Soft Computing. 2016;21(21):1-17

ment. Automatica. 2012;48(6):1077-1087

Conference on. Miami, FL, USA: IEEE; 2017. pp. 1-10

Robotics. 2018;34(1):62-80

2475-1456

May 2010. pp. 4982-4989

22:30-45

#### References


[11] Kantaros Y, Zavlanos M. Distributed communication-aware coverage control by mobile sensor networks. Automatica. 2016;63:209-220

Author details

62 Applications of Mobile Robots

References

36(3):337-354

1-17. ISSN: 1552-3098

trial Electronics. 2017;64(1):750-760

Control. 2018;28(6):2636-2650

Stockholm, Sweden: IEEE; 2016. pp. 4259-4266

Control Council (AACC); June 2013. pp. 6882-6887

Sotiris Papatheodorou and Anthony Tzes\*

\*Address all correspondence to: anthony.tzes@nyu.edu

sensor networks. Automatica. 2017;81:342-349

Transactions on Industrial Informatics. 2013;9(1):451-461

New York University Abu Dhabi, Engineering Division, Abu Dhabi, United Arab Emirates

[1] Pierson A, Figueiredo LC, Pimenta LCA, Schwager M. Adapting to sensing and actuation variations in multi-robot coverage. The International Journal of Robotics Research. 2017;

[2] Abbasi F, Mesbahi A, Velni JM. A team-based approach for coverage control of moving

[3] Mahboubi H, Habibi J, Aghdam AG, Sayrafian-Pour K. Distributed deployment strategies for improved coverage in a network of mobile sensors with prioritized sensing field. IEEE

[4] Palacios-Gasós JM, Montijano E, Sagüés C, Llorente S. Distributed coverage estimation and control for multirobot persistent tasks. IEEE Transactions on Robotics. 2016;PP(99):

[5] Zheng X, Koenig S, Kempe D, Jain S. Multirobot forest coverage for weighted and

[6] Luo C, Yang SX, Li X, Meng MQ-H. Neural-dynamics-driven complete area coverage navigation through cooperation of multiple mobile robots. IEEE Transactions on Indus-

[7] Arslan O, Koditschek DE. Voronoi-based coverage control of heterogeneous disk-shaped robots. In: 2016 IEEE International Conference on Robotics and Automation (ICRA).

[8] Razak RA, Srikant S, Chung H. Decentralized and adaptive control of multiple nonholonomic robots for sensing coverage. International Journal of Robust and Nonlinear

[9] Dirafzoon A, Menhaj MB, Afshar A. Decentralized coverage control for multi-agent systems with nonlinear dynamics. IEICE Transactions on Information and Systems. 2011;94(1):3-10

[10] Mahboubi H, Aghdam AG. Self-deployment algorithms for coverage improvement in a network of nonidentical mobile sensors with limited communication ranges. In: Proceedings American Control Conference (ACC). Washington, DC, USA: American Automatic

unweighted terrain. IEEE Transactions on Robotics. 2010;26(6):1018-1031


[26] Di Franco C, Buttazzo G. Coverage path planning for UAVs photogrammetry with energy and resolution constraints. Journal of Intelligent & Robotic Systems. 2016;83(3-4):1-18

[41] Stergiopoulos Y, Tzes A. Cooperative positioning/orientation control of mobile heterogeneous anisotropic sensor networks for area coverage. In: Proceedings IEEE International Conference on Robotics and Automation (ICRA), IEEE. Hong Kong, China; 2014.

Theoretical and Experimental Collaborative Area Coverage Schemes Using Mobile Agents

http://dx.doi.org/10.5772/intechopen.78940

65

[42] Papatheodorou S, Tzes A, Giannousakis K. Experimental studies on distributed control for area coverage using mobile robots. In: 25th Mediterranean Conference on Control and Automation (MED), Mediterranean Control Association (MCA). Valletta, Malta; July

[43] Garrido-Jurado S, Munoz Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ. Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern

Recognition. 2014;47(6):2280-2292. ISSN: 0031-3203

pp. 1106-1111

2017. pp. 690-695


[41] Stergiopoulos Y, Tzes A. Cooperative positioning/orientation control of mobile heterogeneous anisotropic sensor networks for area coverage. In: Proceedings IEEE International Conference on Robotics and Automation (ICRA), IEEE. Hong Kong, China; 2014. pp. 1106-1111

[26] Di Franco C, Buttazzo G. Coverage path planning for UAVs photogrammetry with energy and resolution constraints. Journal of Intelligent & Robotic Systems. 2016;83(3-4):1-18 [27] Cortés J, Martinez S, Bullo F. Spatially-distributed coverage optimization and control with limited-range interactions. ESAIM: Control, Optimisation and Calculus of Variations.

[28] Kwok A, Martinez S. A distributed deterministic annealing algorithm for limited-range sensor coverage. IEEE Transactions on Control Systems Technology. 2011;19(4):792-804

[29] Kantaros Y, Thanou M, Tzes A. Distributed coverage control for concave areas by a heterogeneous robot-swarm with visibility sensing constraints. Automatica. 2015;53:195-207

[30] Bartolini N, Calamoneri T, La Porta T, Silvestri S. Autonomous deployment of heterogeneous mobile sensors. IEEE Transactions on Mobile Computing. 2011;10(6):753-766 [31] Mahboubi H, Aghdam AG. Distributed deployment algorithms for coverage improvement in a network of wireless mobile sensors: Relocation by virtual force. IEEE Trans-

[32] Flanders H. Differentiation under the integral sign. American Mathematical Monthly.

[33] Habibi J, Mahboubi H, Aghdam AG. Distributed coverage control of mobile sensor networks subject to measurement error. IEEE Transactions on Automatic Control. November

[34] Davis B, Karamouzas I, Guy SJ. C-OPT: Coverage-aware trajectory optimization under uncertainty. IEEE Robotics and Automation Letters. July 2016;1(2):1020-1027. ISSN: 2377-3766 [35] Papatheodorou S, Stergiopoulos Y, Tzes A. Distributed area coverage control with imprecise robot localization. In: 24th Mediterranean Conference on Control and Automation (MED), Mediterranean Control Association (MCA). Athens, Greece; June 2016. pp. 214-219

[36] Evans W, Sember J. Guaranteed Voronoi diagrams of uncertain sites. In: 20th Canadian Conference on Computational Geometry. Montreal, Canada; 2008. pp. 207-210

[37] Gusrialdi A, Hirche S, Asikin D, Hatanaka T, Fujita M. Voronoi-based coverage control with anisotropic sensors and experimental case study. Intelligent Service Robotics. 2009;

[38] Zhang X, Chen X, Liang X, Fang Y. Distributed coverage optimization for deployment of directional sensor networks. In: Decision and Control (CDC), 2015 IEEE 54th Annual

[39] Panagou D, Stipanovic DM, Voulgaris PG. Distributed dynamic coverage and avoidance control under anisotropic sensing. IEEE Transactions on Control of Network Systems.

[40] Laventall K, Cortés J. Coverage control by multi-robot networks with limited-range aniso-

tropic sensory. International Journal of Control. 2009;82(6):1113-1121

Conference on. Osaka, Japan: IEEE; 2015. pp. 246-251

2016;PP(99):1. ISSN: 2325-5870

actions on Control of Network Systems. 2016;61(99):1. ISSN: 2325-5870

2005;11(4):691-719

64 Applications of Mobile Robots

1973;80(6):615-627

2(4):195

2016;61(11):3330-3343. ISSN: 0018-9286


**Chapter 4**

Provisional chapter

**Mobile Robot Feature-Based SLAM Behavior Learning,**

DOI: 10.5772/intechopen.81195

Learning mobile robot space and navigation behavior, are essential requirements for improved navigation, in addition to gain much understanding about the navigation maps. This chapter presents mobile robots feature-based SLAM behavior learning, and navigation in complex spaces. Mobile intelligence has been based on blending a number of functionaries related to navigation, including learning SLAM map main features. To achieve this, the mobile system was built on diverse levels of intelligence, this includes principle component analysis (PCA), neuro-fuzzy (NF) learning system as a classifier, and fuzzy rule based deci-

Keywords: SLAM, PAC, NF classification, fuzzy rule based decision, navigation

Interactive mobile robotics systems have been introduced by researcher's worldwide. The main focus of such research directions, are how to let a mobile robotic system to navigate in an unstructured environment, while learning its features. To meet these objectives, mobile robots platforms are to be equipped with AI tools. In particular to achieve this, Janglová in [1], describes an approach for solving the motion-planning problem in mobile robot control using neural networks-based technique. The proposed system consists of a head artificial neural network, which was used to determine the free space using ultrasound range finder data. In terms of maps building with visual mobile robot capabilities, a remote controlled vision guided mobile robot system was introduced by Raymond et al. [2]. The drive of the

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

Mobile Robot Feature-Based SLAM Behavior Learning,

**and Navigation in Complex Spaces**

and Navigation in Complex Spaces

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.81195

Ebrahim A. Mattar

Ebrahim A. Mattar

Abstract

sion system (FRD).

1. Introduction

1.1. Study background

#### **Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces** Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

DOI: 10.5772/intechopen.81195

Ebrahim A. Mattar Ebrahim A. Mattar

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.81195

#### Abstract

Learning mobile robot space and navigation behavior, are essential requirements for improved navigation, in addition to gain much understanding about the navigation maps. This chapter presents mobile robots feature-based SLAM behavior learning, and navigation in complex spaces. Mobile intelligence has been based on blending a number of functionaries related to navigation, including learning SLAM map main features. To achieve this, the mobile system was built on diverse levels of intelligence, this includes principle component analysis (PCA), neuro-fuzzy (NF) learning system as a classifier, and fuzzy rule based decision system (FRD).

Keywords: SLAM, PAC, NF classification, fuzzy rule based decision, navigation

#### 1. Introduction

#### 1.1. Study background

Interactive mobile robotics systems have been introduced by researcher's worldwide. The main focus of such research directions, are how to let a mobile robotic system to navigate in an unstructured environment, while learning its features. To meet these objectives, mobile robots platforms are to be equipped with AI tools. In particular to achieve this, Janglová in [1], describes an approach for solving the motion-planning problem in mobile robot control using neural networks-based technique. The proposed system consists of a head artificial neural network, which was used to determine the free space using ultrasound range finder data. In terms of maps building with visual mobile robot capabilities, a remote controlled vision guided mobile robot system was introduced by Raymond et al. [2]. The drive of the

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

work, was to describe exploratory research on designing remote controlled emergency stop and vision systems for an autonomous mobile robot. Camera modeling and distortion calibration for mobile robot vision was also introduced by Gang et al. [3]. In their paper they presented an essential camera calibration technique for mobile robot, which is based on PIONEER II experiment platform. Bonin-Font et al. [4], presented a map-based navigation and mapless navigation, as they subdivided in metric map-based navigation and topological map based navigation. Abdul et al. [5] have introduced a hybrid approach for vision based self-localization of autonomous mobile robots. They presented a hybrid approach towards self-localization of tiny autonomous mobile robots in a known but highly dynamic environment. Kalman filter was used for tracking of the globally estimated position. In [8], Filliata and Meyer, presented a 3-level hierarchy of localization strategies, and a direct position inference, single-hypothesis tracking, and multiple-hypothesis tracking.

They stated the advantages and drawbacks of these strategies. In [6], Andreja et al. have presented a fuzzy ART neural architecture for robot map learning and navigation. Araujo proposed methods that are integrated into a navigation architecture. Further, intelligence based navigation was further discussed by [9–11]. In [12], Vlassis et al., motioned, "method for building robot maps by using a Kohonen's self-organizing artificial neural network, and describe how path planning can be subsequently performed on such a map". The built ANN related SOM is shown in Figure 1. Stereo vision-based autonomous mobile robot was also given by Changhan et al. [13]. In their research, they proposed a technique to give more autonomy to a mobile robot by providing vision sensors. In [14], Thrun reported an approach that integrates two paradigms: grid-based and topological. The intelligent control of the mobile robot, was based on image processing was also given by Nima et al. [15]. In terms of leaning intelligent navigation, intelligent robot control using an adaptive critic with a task control center and dynamic database was also introduced by Hall et al. [16]. This involves development and simulation of a real time controller for an intelligent, vision guided robot. Such models are also necessary for sizing the actuators, tuning the controller, and achieving superior performance. A novel feature of the proposed approach is that the method is applicable to both robot arm manipulators and robot bases such as wheeled mobile robots. Stereo vision based self-localization of autonomous mobile robots was furthermore introduced by Abdul et al. [17]. In reference to the work presented, a vision based self-localization of tiny autonomous mobile robots in a known but highly dynamic environment. A learning mobile robots was shown by Hall et al. [18]. They presented a discussion of recent technical advances in learning for intelligent mobile robots. Novel application of a laser range finder with vision system for wheeled mobile robot was presented by Chun et al. [19], where their research presents a trajectory planning strategy of a wheeled mobile robot in an obstructed environment. A vision-based intelligent path following control of a four-wheel differentially driven skid steer mobile robot was given by Nazari and Naraghi [20]. In this work, a Fuzzy Logic Controller (FLC) for path following of a four-wheel differentially skid steer mobile robot is presented. Color learning and illumination invariance on mobile robots survey was given by Mohan et al. [21]. A major challenge to the widespread deployment of mobile robots is the ability to function autonomously. Two arms and two legs like an ape, was aimed to study a variety of vision-based behaviors. In addition, robot based on the remote-brained approach

was given by Masayuki et al. [22]. Localization algorithm of mobile robot based on single vision and laser radar have been presented by Xiaoning [23]. In order to increase the localization precision of mobile robot, a self-localization algorithm based on odometry, single vision and laser radar is proposed. The data provided by odometry, single vision, and laser radar were fused together by means of an Extended Kalman filter (EKF) technique. Mobile robot selflocalization in complex indoor environments using monocular vision and 3D model, was moreover presented by Andreja et al. [24], they considered the problem of mobile robot pose estimation using only visual information from a single camera and odometry readings. Human observation based mobile robot navigation in intelligent space was also given by

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

http://dx.doi.org/10.5772/intechopen.81195

69

Figure 1. ANN-self organizing maps, for learning mobile robot navigation [1, 6–8].

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces http://dx.doi.org/10.5772/intechopen.81195 69

Figure 1. ANN-self organizing maps, for learning mobile robot navigation [1, 6–8].

work, was to describe exploratory research on designing remote controlled emergency stop and vision systems for an autonomous mobile robot. Camera modeling and distortion calibration for mobile robot vision was also introduced by Gang et al. [3]. In their paper they presented an essential camera calibration technique for mobile robot, which is based on PIONEER II experiment platform. Bonin-Font et al. [4], presented a map-based navigation and mapless navigation, as they subdivided in metric map-based navigation and topological map based navigation. Abdul et al. [5] have introduced a hybrid approach for vision based self-localization of autonomous mobile robots. They presented a hybrid approach towards self-localization of tiny autonomous mobile robots in a known but highly dynamic environment. Kalman filter was used for tracking of the globally estimated position. In [8], Filliata and Meyer, presented a 3-level hierarchy of localization strategies, and a direct position inference, single-hypothesis tracking,

They stated the advantages and drawbacks of these strategies. In [6], Andreja et al. have presented a fuzzy ART neural architecture for robot map learning and navigation. Araujo proposed methods that are integrated into a navigation architecture. Further, intelligence based navigation was further discussed by [9–11]. In [12], Vlassis et al., motioned, "method for building robot maps by using a Kohonen's self-organizing artificial neural network, and describe how path planning can be subsequently performed on such a map". The built ANN related SOM is shown in Figure 1. Stereo vision-based autonomous mobile robot was also given by Changhan et al. [13]. In their research, they proposed a technique to give more autonomy to a mobile robot by providing vision sensors. In [14], Thrun reported an approach that integrates two paradigms: grid-based and topological. The intelligent control of the mobile robot, was based on image processing was also given by Nima et al. [15]. In terms of leaning intelligent navigation, intelligent robot control using an adaptive critic with a task control center and dynamic database was also introduced by Hall et al. [16]. This involves development and simulation of a real time controller for an intelligent, vision guided robot. Such models are also necessary for sizing the actuators, tuning the controller, and achieving superior performance. A novel feature of the proposed approach is that the method is applicable to both robot arm manipulators and robot bases such as wheeled mobile robots. Stereo vision based self-localization of autonomous mobile robots was furthermore introduced by Abdul et al. [17]. In reference to the work presented, a vision based self-localization of tiny autonomous mobile robots in a known but highly dynamic environment. A learning mobile robots was shown by Hall et al. [18]. They presented a discussion of recent technical advances in learning for intelligent mobile robots. Novel application of a laser range finder with vision system for wheeled mobile robot was presented by Chun et al. [19], where their research presents a trajectory planning strategy of a wheeled mobile robot in an obstructed environment. A vision-based intelligent path following control of a four-wheel differentially driven skid steer mobile robot was given by Nazari and Naraghi [20]. In this work, a Fuzzy Logic Controller (FLC) for path following of a four-wheel differentially skid steer mobile robot is presented. Color learning and illumination invariance on mobile robots survey was given by Mohan et al. [21]. A major challenge to the widespread deployment of mobile robots is the ability to function autonomously. Two arms and two legs like an ape, was aimed to study a variety of vision-based behaviors. In addition, robot based on the remote-brained approach

and multiple-hypothesis tracking.

68 Applications of Mobile Robots

was given by Masayuki et al. [22]. Localization algorithm of mobile robot based on single vision and laser radar have been presented by Xiaoning [23]. In order to increase the localization precision of mobile robot, a self-localization algorithm based on odometry, single vision and laser radar is proposed. The data provided by odometry, single vision, and laser radar were fused together by means of an Extended Kalman filter (EKF) technique. Mobile robot selflocalization in complex indoor environments using monocular vision and 3D model, was moreover presented by Andreja et al. [24], they considered the problem of mobile robot pose estimation using only visual information from a single camera and odometry readings. Human observation based mobile robot navigation in intelligent space was also given by Takeshi and Hashimoto [25], they investigated a mobile robot navigation system which can localize the mobile robot correctly and navigate based on observation of human walking. Similar work was also given by Manoj and Ernest [26].

1.2. Research objectives

2. Building navigation maps

Optimal path search is done by A<sup>∗</sup>

processing using the PCA algorithm.

2.2. Monte-Carlo (MC) localization

xk ¼

2.1. Simultaneous localization and mapping (SLAM)

Given the previous background, this current presented work is focusing on learning navigation maps with intelligent capabilities. The system is based on PCA representation of large navigation maps, Neuro-fuzzy classifier, and a fuzzy decision based system. For bulky amount of visual and non-visual mobile data measurements (odometry, and the observations), the approach followed, is to reduce the mobile robot observation dimensionality using principle component analysis (PCA), thus to generate a reduced representation of the navigation map (SLAM), refer to Figure 2 for details. A learning system was used to learn navigation maps details, hence to classify the representations, (in terms of observation features). The learned system was

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

http://dx.doi.org/10.5772/intechopen.81195

71

SLAM, is a routine that estimates a pose of a mobile robot, while mapping the environment at the same time. SLAM is computationally intensive, since maps represent localization, hence accurate pose estimate is needed for mapping. For creating navigation intelligence capabilities during path planning, this is achieved by learning spaces, once robot was in motion. This is based on learning path and navigation behavior. There are four important stages for building SLAM, this localization, map building and updates, searching for optimal path and planning.

inputs U as set of controls, U<sup>1</sup>:<sup>k</sup> ¼ ð Þ u1; u2;…; uk , with mobile parameters measurements

0 B@

0 B@

In reference to Figure 3, the mobile robot starts to move in the space, with a target to research into a predefined final position. While the robot in movement, a SLAM is built, and measurements are recorded from the mobile observations, as Z<sup>1</sup>:<sup>k</sup> ¼ ð Þ z1; z2;…; zk . All recorded observations are considered as inputs to the PAC, hence they are tabulated into predefined format, for later

Monte-Carlo localization, is a well-known technique in literature, and still being used for localization parameters estimation. In sampling-based methods, one represents the density by

(mobile observations) as Z<sup>1</sup>:<sup>k</sup> ¼ ð Þ z1; z2;…; zk . For odometry, refer to Figure 3.

and Ck ¼

xr

1

CCCCCCCCCCCCA

0

BBBBBBBBBBBB@

m<sup>1</sup>

m<sup>2</sup> : : mn

, occupancy grids mapping. Given a mobile robot control

1 CA

1

CA (1)

Cr ⋯ CRMn ⋮⋱⋮ CMrR ⋯ CMn

employed for navigating maps, and other mobile robot routing applications.

Figure 2. (a) A sample of SLAM details for learning, picture source robot cartography [27]. (b) Mobile robot training patterns generation. A space is represented by maps space's basis.

#### 1.2. Research objectives

Takeshi and Hashimoto [25], they investigated a mobile robot navigation system which can localize the mobile robot correctly and navigate based on observation of human walking.

Figure 2. (a) A sample of SLAM details for learning, picture source robot cartography [27]. (b) Mobile robot training

patterns generation. A space is represented by maps space's basis.

Similar work was also given by Manoj and Ernest [26].

70 Applications of Mobile Robots

Given the previous background, this current presented work is focusing on learning navigation maps with intelligent capabilities. The system is based on PCA representation of large navigation maps, Neuro-fuzzy classifier, and a fuzzy decision based system. For bulky amount of visual and non-visual mobile data measurements (odometry, and the observations), the approach followed, is to reduce the mobile robot observation dimensionality using principle component analysis (PCA), thus to generate a reduced representation of the navigation map (SLAM), refer to Figure 2 for details. A learning system was used to learn navigation maps details, hence to classify the representations, (in terms of observation features). The learned system was employed for navigating maps, and other mobile robot routing applications.

#### 2. Building navigation maps

#### 2.1. Simultaneous localization and mapping (SLAM)

SLAM, is a routine that estimates a pose of a mobile robot, while mapping the environment at the same time. SLAM is computationally intensive, since maps represent localization, hence accurate pose estimate is needed for mapping. For creating navigation intelligence capabilities during path planning, this is achieved by learning spaces, once robot was in motion. This is based on learning path and navigation behavior. There are four important stages for building SLAM, this localization, map building and updates, searching for optimal path and planning. Optimal path search is done by A<sup>∗</sup> , occupancy grids mapping. Given a mobile robot control inputs U as set of controls, U<sup>1</sup>:<sup>k</sup> ¼ ð Þ u1; u2;…; uk , with mobile parameters measurements (mobile observations) as Z<sup>1</sup>:<sup>k</sup> ¼ ð Þ z1; z2;…; zk . For odometry, refer to Figure 3.

$$\mathbf{x}\_k = \begin{pmatrix} \mathbf{x}\_r \\\\ m\_1 \\\\ m\_2 \\\\ \vdots \\\\ m\_n \end{pmatrix} \text{ and } \mathbf{C}\_k = \left( \begin{pmatrix} \mathbf{C}\_r & \cdots & \mathbf{C}\_{RM\_\pi} \\\\ \vdots & \ddots & \vdots \\\\ \mathbf{C}\_{M,\mathbb{R}} & \cdots & \mathbf{C}\_{M\_\pi} \end{pmatrix} \right) \tag{1}$$

In reference to Figure 3, the mobile robot starts to move in the space, with a target to research into a predefined final position. While the robot in movement, a SLAM is built, and measurements are recorded from the mobile observations, as Z<sup>1</sup>:<sup>k</sup> ¼ ð Þ z1; z2;…; zk . All recorded observations are considered as inputs to the PAC, hence they are tabulated into predefined format, for later processing using the PCA algorithm.

#### 2.2. Monte-Carlo (MC) localization

Monte-Carlo localization, is a well-known technique in literature, and still being used for localization parameters estimation. In sampling-based methods, one represents the density by

Figure 3. Odometry, generation of navigation patterns for spaces, the (maps).

a set of η random samples of particles. The goal is then to recursively compute at each time step k set of samples of Ζ<sup>k</sup> that is drawn from density p xk : Zk � �. A particularly elegant algorithm to accomplish this has recently been suggested independently by various authors. In analogy with the formal filtering problem, the algorithm proceeds in two phases. In the first phase we start from a set of particles Sð Þ <sup>k</sup>�<sup>1</sup> computed in the previous iteration, and apply the motion model to each particle Si ð Þ <sup>k</sup>�<sup>1</sup> by sampling from the density <sup>p</sup>ðxk⋮Si ð Þ <sup>k</sup>�<sup>1</sup> , uð Þ <sup>k</sup>�<sup>1</sup> <sup>Þ</sup> for each particle S<sup>i</sup> ð Þ <sup>k</sup>�<sup>1</sup> : draw one sample Si ð Þ<sup>k</sup> from <sup>ð</sup>xk⋮S<sup>i</sup> ð Þ <sup>k</sup>�<sup>1</sup> , uð Þ <sup>k</sup>�<sup>1</sup> <sup>Þ</sup>. We have used a motion model and set of particles S<sup>i</sup> ð Þ <sup>k</sup>�<sup>1</sup> to build an empirical predictive density function of:

$$(p)' = f(\mathbf{x}, y, \theta, \Delta \mathbf{s}\_r, \Delta \mathbf{s}\_l), \\
(p)' = \begin{pmatrix} \mathbf{x} \\ \mathbf{y} \\ \theta \end{pmatrix} + \begin{pmatrix} \frac{\Delta \mathbf{s}\_r + \Delta \mathbf{s}\_l}{2} \cos \left(\theta + \frac{\Delta \mathbf{s}\_r - \Delta \mathbf{s}\_l}{2b}\right) \\\ \frac{\Delta \mathbf{s}\_r + \Delta \mathbf{s}\_l}{2} \sin \left(\theta + \frac{\Delta \mathbf{s}\_r - \Delta \mathbf{s}\_l}{2b}\right) \\\ \frac{\Delta \mathbf{s}\_r - \Delta \mathbf{s}\_l}{b} \end{pmatrix} \tag{2}$$

each location during navigation, a covariance matrix for the set of maps is considered highly

Figure 4. Robot navigation spaces. A representation of navigation segments, zones, areas, ….. (S1, S2, S3, …. Sn, Z1, Z2, Z3,

<sup>11</sup> ⋯ σ<sup>X</sup>

⋮⋱ ⋮

<sup>λ</sup><sup>n</sup> � <sup>λ</sup><sup>T</sup> n

, since the <sup>P</sup>'s columns are orthonormal to each other, PT � <sup>P</sup> <sup>¼</sup> <sup>I</sup>. Now, the

<sup>w</sup>�h, <sup>1</sup> <sup>⋯</sup> <sup>σ</sup><sup>X</sup>

ij represents covariance between distances for location (w) and location (h). There is a relation between covariance coefficients and correlation coefficients. The covariance matrix r is expressed as:

1,w�h

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

http://dx.doi.org/10.5772/intechopen.81195

73

1 CA

1

� � (4)

CA (3)

w�h,w�h

σX

σX

<sup>r</sup> <sup>¼</sup> <sup>1</sup> j X j

Sy <sup>¼</sup> PT � Sy � <sup>P</sup>

Since principal components are calculated linearly, let Ρ be a transformation matrix:

n¼1

<sup>Y</sup> <sup>¼</sup> <sup>P</sup><sup>T</sup> � X and X <sup>¼</sup> <sup>P</sup> � <sup>Y</sup>

question is, what is the value of Ρ given the condition that Sy must be a diagonal matrix, i.e.

Sy <sup>¼</sup> <sup>Y</sup> � <sup>Y</sup><sup>T</sup> <sup>¼</sup> PT � <sup>Χ</sup> � <sup>Χ</sup><sup>T</sup> � <sup>P</sup>

in such away Sy is a rotation of Sx by Ρ. ChoosingΡ as being a matrix containing eigenvectors of Sx:

0 B@

0 B@

non-diagonal. Mathematically, the previous notation is:

σX

In fact, <sup>P</sup> <sup>¼</sup> <sup>P</sup>�<sup>1</sup>

…. Zm, A1, A2, A3, …. An).

<sup>r</sup> <sup>¼</sup> XXt

In Eq. (2), we describe a blended density approximation to p xk⋮Zð Þ <sup>k</sup>�<sup>1</sup> � �. The environment, or the mobile robot space is highly redundant, once used to describe maps.

#### 3. Principle component analysis (PCA)

#### 3.1. PCA based statistically and dimensionality reduction

While in navigation, each traveled path, region, zone, etc. are characterized by diverse behavior (i.e. features), Figure 4. If x is matrix of representation for distances and measurements at

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces http://dx.doi.org/10.5772/intechopen.81195 73

Figure 4. Robot navigation spaces. A representation of navigation segments, zones, areas, ….. (S1, S2, S3, …. Sn, Z1, Z2, Z3, …. Zm, A1, A2, A3, …. An).

a set of η random samples of particles. The goal is then to recursively compute at each time step k set of samples of Ζ<sup>k</sup> that is drawn from density p xk : Zk � �. A particularly elegant algorithm to accomplish this has recently been suggested independently by various authors. In analogy with the formal filtering problem, the algorithm proceeds in two phases. In the first phase we start from a set of particles Sð Þ <sup>k</sup>�<sup>1</sup> computed in the previous iteration, and apply the

ð Þ<sup>k</sup> from <sup>ð</sup>xk⋮S<sup>i</sup>

ð Þ <sup>k</sup>�<sup>1</sup> to build an empirical predictive density function of:

x y θ 1

CA <sup>þ</sup>

While in navigation, each traveled path, region, zone, etc. are characterized by diverse behavior (i.e. features), Figure 4. If x is matrix of representation for distances and measurements at

0 B@

In Eq. (2), we describe a blended density approximation to p xk⋮Zð Þ <sup>k</sup>�<sup>1</sup> � �

the mobile robot space is highly redundant, once used to describe maps.

ð Þ <sup>k</sup>�<sup>1</sup> by sampling from the density <sup>p</sup>ðxk⋮Si

0

BBBBBBB@

Δsr þ Δsl

Δsr þ Δsl

<sup>2</sup> cos <sup>θ</sup> <sup>þ</sup>

<sup>2</sup> sin <sup>θ</sup> <sup>þ</sup>

Δsr � Δsl b

ð Þ <sup>k</sup>�<sup>1</sup> , uð Þ <sup>k</sup>�<sup>1</sup> <sup>Þ</sup> for each

1

CCCCCCCA

. The environment, or

(2)

ð Þ <sup>k</sup>�<sup>1</sup> , uð Þ <sup>k</sup>�<sup>1</sup> <sup>Þ</sup>. We have used a motion model and

Δsr � Δsl 2b � �

Δsr � Δsl 2b � �

motion model to each particle Si

ð Þ <sup>k</sup>�<sup>1</sup> : draw one sample Si

ð Þp <sup>0</sup> ¼ f x; y; θ; Δsr ð Þ ; Δsl , pð Þ<sup>0</sup> ¼

Figure 3. Odometry, generation of navigation patterns for spaces, the (maps).

3. Principle component analysis (PCA)

3.1. PCA based statistically and dimensionality reduction

particle S<sup>i</sup>

set of particles S<sup>i</sup>

72 Applications of Mobile Robots

each location during navigation, a covariance matrix for the set of maps is considered highly non-diagonal. Mathematically, the previous notation is:

$$\boldsymbol{\rho} = \mathbf{X} \mathbf{X}^{t} \left( \begin{pmatrix} \sigma\_{11}^{\mathbf{X}} & \cdots & \sigma\_{1,w \times h}^{\mathbf{X}} \\ \vdots & \ddots & \vdots \\ \sigma\_{w \times h, 1}^{\mathbf{X}} & \cdots & \sigma\_{w \times h, w \times h}^{\mathbf{X}} \end{pmatrix} \right) \tag{3}$$

σX ij represents covariance between distances for location (w) and location (h). There is a relation between covariance coefficients and correlation coefficients. The covariance matrix r is expressed as:

$$\rho = \frac{1}{j} \sum\_{n=1}^{j} \left(\lambda\_n \times \lambda\_n^T\right) \tag{4}$$

Since principal components are calculated linearly, let Ρ be a transformation matrix:

$$Y = P^T \times X \quad \text{and} \quad X = P \times Y$$

In fact, <sup>P</sup> <sup>¼</sup> <sup>P</sup>�<sup>1</sup> , since the <sup>P</sup>'s columns are orthonormal to each other, PT � <sup>P</sup> <sup>¼</sup> <sup>I</sup>. Now, the question is, what is the value of Ρ given the condition that Sy must be a diagonal matrix, i.e.

$$\begin{aligned} \mathbf{S\_y} &= \mathbf{Y} \times \mathbf{Y}^T = \boldsymbol{P}^T \times \mathbf{X} \times \mathbf{X}^T \times \boldsymbol{P} \\ \mathbf{S\_y} &= \boldsymbol{P}^T \times \mathbf{S\_y} \times \boldsymbol{P} \end{aligned}$$

in such away Sy is a rotation of Sx by Ρ. ChoosingΡ as being a matrix containing eigenvectors of Sx:

$$\mathcal{S}\_{\mathfrak{x}} \times P = \Lambda \times P$$

where Λ is a diagonal matrix containing eigenvalues of Sx. In this regard,

$$S\_y = P^T \times \Lambda \times P = \Lambda \times P^T \times P = \Lambda$$

and Sy is a diagonal matrix containing eigenvalues of Sx. Since the diagonal elements of Sy are the variance of components of training patterns in the (in navigation) space, the eigenvalues of Sx are those variances.

This is further expanded into:

$$
\rho = \begin{pmatrix}
\beta\_{1,1}\overline{\beta}\_1 & \cdots & \beta\_{1,k}\overline{\beta}\_k \\
\vdots & \ddots & \vdots \\
\beta\_{j,1}\overline{\beta}\_1 & \cdots & \beta\_{j,k}\overline{\pi}\_k
\end{pmatrix} \* \begin{pmatrix}
\beta\_{1,1}\overline{\beta}\_1 & \cdots & \beta\_{j,1}\overline{\beta}\_1 \\
\vdots & \ddots & \vdots \\
\beta\_{1,k}\overline{\beta}\_k & \cdots & \beta\_{j,k}\overline{\beta}\_k
\end{pmatrix} \tag{5}
$$

Finally, covariance matrix r is further expressed by:

$$\rho = \begin{pmatrix} cov(\beta\_1, \beta\_1) & \cdots & cov(\beta\_1, \beta\_k) \\ \vdots & \ddots & \vdots \\ cov(\beta\_k, \beta\_1) & \cdots & cov(\beta\_k, \beta\_k) \end{pmatrix} \tag{6}$$

Following Eq. (5) to Eq. (9), and computing for eigenvalues, hence reordering the eigenvalues according to a descending order, this represents a major step in building a PAC based recog-

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

http://dx.doi.org/10.5772/intechopen.81195

75

While stating in Section 3 steps for PAC computations, in this section we shall make a focus on building a learning system for the mobile navigation. In reference to Figure 4, the mobile robot will be generating navigation dataset. Dataset is generated at different locations, during the mobile robot motion. Navigation dataset involves sensory measurements, odometry, and locality information (i.e. zones, areas, segments …), refer to Figure 4. An important part of the dataset, is also the part generated by the SLAM, as already described in Section 2. It is not achievable to encompass all dataset, as this is massive dataset. However, we shall rely on features of the mobile dataset, i.e. the PCA based features of navigation dataset, (the SLAM features). Robot navigation spaces. A representation of navigation segments, zones, areas, ….. are designated as (S1, S2, S3, …. Sn, Z1, Z2, Z3, …. Zm, A1, A2, A3, …. An). For each of such different segments, zones, and areas of navigation, there will be features associated with it. Features during navigation will be used for further processing. This includes a five layers feature learning NF architecture (classifier), and a fuzzy decision

For the case of fuzzy decision making system, it is essential to incorporate a priori knowledge for the mobile movements in the space. In this respect, many conventional approaches rely on depth physical knowledge describing the system. An issue with fuzzy decision is that knowledge are mathematically impervious. This results in and there is no formal mathematical representation of the system's behavior. This prevents the application of conventional empirical modeling techniques to fuzzy systems, making knowledge validation and comparison hard to perform. Creating a benchmark measure of performance by a minimum distance classifier. Decision rule adopted by such system, is to assign Χ to a class whose mean feature vector is

∥d � d1∥ ≤ ∥d � d2¼)d∈ h<sup>2</sup>

Rule-based structure of fuzzy knowledge allows for integrating heuristic knowledge with information obtained from process measurements. The global operation of a system is divided

into several local operating conditions. Within each region Ri, a representation:

else d∈h<sup>2</sup>

(10)

nition system for the mobile robot dataset generated by the navigation system.

4. Learning system: learning mobile robot navigation maps

4.1. Feature-based SLAM learning

system (Figure 5).

4.2. Neuro-fuzzy features classifier (NFC) architecture

closest (Euclidean Distance) to Χ. A decision is given by:

(

The p, is a symmetric (around the main diagonal), and a R<sup>n</sup>�<sup>n</sup> matrix. The diagonal of, represents the covariance between the matrix and it self. To recognize normalized dataset for patterns, the covariance r of Eq. (6) plays an important role. This can be achieved by getting the eigenvectors of covariance r matrix of Eq. (6). Given this background, therefore we need to compute eigenvalues and eigenvectors using numerical approach. For a R<sup>k</sup>�<sup>k</sup> matrix r, if we search for a row vector R<sup>k</sup>�<sup>1</sup> X that could be multiplied by r and get the same vector X multiplied by eigenvalues λ and eigenvector. Matrix r transforms the vector X to scale positions by an amount equal to λ, gives a transformation matrix:

$$\mathbf{u}(\rho \mathbf{X}) = (\lambda \mathbf{X}) \tag{7}$$

In reference to Eq. (7), and for a R<sup>k</sup>�<sup>k</sup> matrix r, we shall compute for (k) eigenvalues. The (k) eigenvalue (λ), are hence used for scaling every (k) eigenvectors. Individual eigenvalues (λ), are also found by solving the below defined identity as expressed by Eq. (8):

$$[(\rho - I \times \lambda)\mathbf{X} = \mathbf{0}] \tag{8}$$

where I is an identity matrix. We shall compute for the determinant of Eq. (8), i.e., |ð Þ r � Iλ ∣ ¼ 0 , while solving for the eigenvalues, λ. While substituting for the (λ) in Eq. (8), and solving for (X), this will result in finding the eigenvector (X), once λ are satisfying the following:

$$|(\rho - I \times \lambda)| = 0 \tag{9}$$

Following Eq. (5) to Eq. (9), and computing for eigenvalues, hence reordering the eigenvalues according to a descending order, this represents a major step in building a PAC based recognition system for the mobile robot dataset generated by the navigation system.

#### 4. Learning system: learning mobile robot navigation maps

#### 4.1. Feature-based SLAM learning

Sx � P ¼ Λ � P

Sy <sup>¼</sup> <sup>P</sup><sup>T</sup> � <sup>Λ</sup> � <sup>P</sup> <sup>¼</sup> <sup>Λ</sup> � <sup>P</sup><sup>T</sup> � <sup>P</sup> <sup>¼</sup> <sup>Λ</sup>

and Sy is a diagonal matrix containing eigenvalues of Sx. Since the diagonal elements of Sy are the variance of components of training patterns in the (in navigation) space, the eigenvalues of

1

0 B@

� � <sup>⋯</sup> cov <sup>β</sup>1; <sup>β</sup><sup>k</sup>

� � <sup>⋯</sup> cov <sup>β</sup>k; <sup>β</sup><sup>k</sup>

⋮⋱⋮

The p, is a symmetric (around the main diagonal), and a R<sup>n</sup>�<sup>n</sup> matrix. The diagonal of, represents the covariance between the matrix and it self. To recognize normalized dataset for patterns, the covariance r of Eq. (6) plays an important role. This can be achieved by getting the eigenvectors of covariance r matrix of Eq. (6). Given this background, therefore we need to compute eigenvalues and eigenvectors using numerical approach. For a R<sup>k</sup>�<sup>k</sup> matrix r, if we search for a row vector R<sup>k</sup>�<sup>1</sup> X that could be multiplied by r and get the same vector X multiplied by eigenvalues λ and eigenvector. Matrix r transforms the vector X to scale posi-

In reference to Eq. (7), and for a R<sup>k</sup>�<sup>k</sup> matrix r, we shall compute for (k) eigenvalues. The (k) eigenvalue (λ), are hence used for scaling every (k) eigenvectors. Individual eigenvalues (λ),

where I is an identity matrix. We shall compute for the determinant of Eq. (8), i.e., |ð Þ r � Iλ ∣ ¼ 0 , while solving for the eigenvalues, λ. While substituting for the (λ) in Eq. (8), and solving for (X), this will result in finding the eigenvector (X), once λ are satisfying the

are also found by solving the below defined identity as expressed by Eq. (8):

CA∗

<sup>β</sup>1,1�β<sup>1</sup> <sup>⋯</sup> <sup>β</sup>j,1�β<sup>1</sup> ⋮⋱⋮ <sup>β</sup>1, <sup>k</sup>�β<sup>k</sup> <sup>⋯</sup> <sup>β</sup>j, <sup>k</sup>�β<sup>k</sup>

� �

1

ð Þ¼ rX ð Þ λX (7)

½ � ð Þ r � I � λ X ¼ 0 (8)

j j ð Þ r � I � λ ¼ 0 (9)

� �

1

CA (5)

CA (6)

where Λ is a diagonal matrix containing eigenvalues of Sx. In this regard,

<sup>β</sup>1,1�β<sup>1</sup> <sup>⋯</sup> <sup>β</sup>1, <sup>k</sup>�β<sup>k</sup> ⋮⋱⋮ <sup>β</sup>j,1�β<sup>1</sup> <sup>⋯</sup> <sup>β</sup>j, <sup>k</sup>�xk

cov β1; β<sup>1</sup>

cov βk; β<sup>1</sup>

Sx are those variances.

74 Applications of Mobile Robots

following:

This is further expanded into:

r ¼

0 B@

Finally, covariance matrix r is further expressed by:

r ¼

tions by an amount equal to λ, gives a transformation matrix:

0 B@ While stating in Section 3 steps for PAC computations, in this section we shall make a focus on building a learning system for the mobile navigation. In reference to Figure 4, the mobile robot will be generating navigation dataset. Dataset is generated at different locations, during the mobile robot motion. Navigation dataset involves sensory measurements, odometry, and locality information (i.e. zones, areas, segments …), refer to Figure 4. An important part of the dataset, is also the part generated by the SLAM, as already described in Section 2. It is not achievable to encompass all dataset, as this is massive dataset. However, we shall rely on features of the mobile dataset, i.e. the PCA based features of navigation dataset, (the SLAM features). Robot navigation spaces. A representation of navigation segments, zones, areas, ….. are designated as (S1, S2, S3, …. Sn, Z1, Z2, Z3, …. Zm, A1, A2, A3, …. An). For each of such different segments, zones, and areas of navigation, there will be features associated with it. Features during navigation will be used for further processing. This includes a five layers feature learning NF architecture (classifier), and a fuzzy decision system (Figure 5).

#### 4.2. Neuro-fuzzy features classifier (NFC) architecture

For the case of fuzzy decision making system, it is essential to incorporate a priori knowledge for the mobile movements in the space. In this respect, many conventional approaches rely on depth physical knowledge describing the system. An issue with fuzzy decision is that knowledge are mathematically impervious. This results in and there is no formal mathematical representation of the system's behavior. This prevents the application of conventional empirical modeling techniques to fuzzy systems, making knowledge validation and comparison hard to perform. Creating a benchmark measure of performance by a minimum distance classifier.

Decision rule adopted by such system, is to assign Χ to a class whose mean feature vector is closest (Euclidean Distance) to Χ. A decision is given by:

$$\begin{cases} \|d - \overline{d}\_1\| & \le \quad \|d - \overline{d}\_2 \Longrightarrow d \in h\_2\\ & \text{else} \qquad \qquad d \in h\_2 \end{cases} \tag{10}$$

Rule-based structure of fuzzy knowledge allows for integrating heuristic knowledge with information obtained from process measurements. The global operation of a system is divided into several local operating conditions. Within each region Ri, a representation:

Figure 5. The recognition system. PAC for SLAM features computations, a five layers neuro-fuzzy classifier, and last a fuzzy decision based system.

$$R\_i \hat{y}\_i(k) = \sum\_{\vec{\imath}\,\vec{\jmath}}^o \chi\_{\vec{\imath}\,\vec{\jmath}} y(k) + \sum\_{\vec{\imath}\,\vec{\jmath}}^h \psi\_{\vec{\imath}\,\vec{\jmath}} u(k) \text{ for } \mathbf{h} = 1, 2, \dots \text{r} \tag{11}$$

μijðχsjÞ is the membership grade of i

where νsj is the input to i

ranged between (1 and 0).

For each i

th rule and j

<sup>α</sup>is <sup>¼</sup> <sup>Y</sup><sup>n</sup> j¼1

th node in this layer is a square node with a node function.

Figure 6. NF classifier architecture. The architecture is used to classify features of navigation maps.

in the ordinary fuzzy (if-then) rules, (then) parts are some combinations of the input variables.

this node function, n is the number of features. The membership is a bell-shape type, and

As values of these parameters change, membership shaped functions vary accordingly, thus exhibiting various forms of membership functions on the linguistic label Ai. For every node in

Osk <sup>¼</sup> <sup>β</sup> P sk k <sup>l</sup>¼<sup>1</sup> <sup>β</sup>sl

th. That is, the (if) parts of the rules are same as

<sup>μ</sup>ij <sup>ν</sup>sj � � (13)

(14)

th node given as the linguistic label (small, large, .. etc.) associated with

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

http://dx.doi.org/10.5772/intechopen.81195

77

In reference to Figure 6, for Eq. (11), y^<sup>i</sup> is the computed fuzzy output, u is the system input, in the i th operating region, (h) is the number of fuzzy operating regions. In addition, both (i) and (o) do represent the time lags in the input and the output, respectively, μ<sup>i</sup> is the membership function. Finally, χij and ψij are the few parameters. The membership function for inputs, is constructed in a number of ways.

The fuzzy knowledge system (Neuro-fuzzy) illustrated in Figure 6, is an exceptional architecture of network topology. This architecture combines advantages of fuzzy reasoning and the classical neural networks. In its broader sense, the architecture rule ð Þ ri , demonstrates a relation between the input map feature space, and named classes. This is further expressed as follows:

$$\text{Rule } \mathbf{r}\_{i} \implies \text{if } \quad \chi\_{si} \text{ and } \ \chi\_{s\not\gamma} \text{ is } A\_{i\not\gamma} \dots \dots \text{ and } \ \chi\_{sn} \text{ is } A\_{in\nu} \implies \text{class name is } \mathbb{C}\_{k} \tag{12}$$

In Eq. (12), the Gaussian membership function is defined as:

$$\mu\_{i\circ} \left( \chi\_{s\circ} \right) = \exp \left( - \left( \chi\_{s\circ} - c\_{i\circ} \right)^2 \vert 2\sigma\_{i\circ}^2 \rangle \right)$$

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces http://dx.doi.org/10.5772/intechopen.81195 77

Figure 6. NF classifier architecture. The architecture is used to classify features of navigation maps.

Riy^<sup>i</sup>

constructed in a number of ways.

fuzzy decision based system.

76 Applications of Mobile Robots

the i

follows:

ð Þ¼ <sup>k</sup> <sup>X</sup><sup>o</sup> ij

In Eq. (12), the Gaussian membership function is defined as:

μij χsj � �

<sup>χ</sup>ijy kð Þþ<sup>X</sup>

h

Figure 5. The recognition system. PAC for SLAM features computations, a five layers neuro-fuzzy classifier, and last a

ij

In reference to Figure 6, for Eq. (11), y^<sup>i</sup> is the computed fuzzy output, u is the system input, in

The fuzzy knowledge system (Neuro-fuzzy) illustrated in Figure 6, is an exceptional architecture of network topology. This architecture combines advantages of fuzzy reasoning and the classical neural networks. In its broader sense, the architecture rule ð Þ ri , demonstrates a relation between the input map feature space, and named classes. This is further expressed as

¼ exp � χsj � cij

Rule ri ¼)if χsi and χsj is Aij…… and χsn is Ain, ¼)class name is Ck (12)

� �<sup>2</sup>

� �

<sup>j</sup>2σ<sup>2</sup> ij

th operating region, (h) is the number of fuzzy operating regions. In addition, both (i) and (o) do represent the time lags in the input and the output, respectively, μ<sup>i</sup> is the membership function. Finally, χij and ψij are the few parameters. The membership function for inputs, is

ψiju kð Þ for h ¼ 1, 2, …r (11)

μijðχsjÞ is the membership grade of i th rule and j th. That is, the (if) parts of the rules are same as in the ordinary fuzzy (if-then) rules, (then) parts are some combinations of the input variables. For each i th node in this layer is a square node with a node function.

$$\alpha\_{\dot{s}\dot{s}} = \prod\_{j=1}^{n} \mu\_{ij} \left( \nu\_{\dot{s}\dot{\cdot}} \right) \tag{13}$$

where νsj is the input to i th node given as the linguistic label (small, large, .. etc.) associated with this node function, n is the number of features. The membership is a bell-shape type, and ranged between (1 and 0).

$$O\_{sk} = \frac{\beta\_{sk}}{\sum\_{l=1}^{k} \beta\_{sl}} \tag{14}$$

As values of these parameters change, membership shaped functions vary accordingly, thus exhibiting various forms of membership functions on the linguistic label Ai. For every node in this layer, there is a circle node which multiplies incoming signals and sends their product out. Stages of the adopted Neuro-fuzzy classifier, is shown in Figure 6.

$$\chi\_i = \mu A\_i \mathbf{x}(k\_1) \times \mu B\_i y(k\_2) \text{ for } i = 1, 2 \tag{15}$$

The output node computes the system output as summation of incoming signals, i.e.:

$$X\_i^o = \sum\_i \overline{Y}\_{\dot{\jmath}} f\_i \to X\_i^o = \frac{\sum\_i \overline{Y}\_{\dot{\jmath}} f\_i}{\sum\_i \overline{Y}\_i} \tag{16}$$

More precisely, the class label for the s th sample is obtained by the maximum f g Osk value as follows:

$$\mathbb{C}\_s = \max\_{k=1,2,\ldots,K} \{ \mathbf{O}\_{\text{sk}} \} \tag{17}$$

The consequent parameters thus identified are optimal (in the consequent parameter space) under the condition that the premise parameters are fixed. The knowledge system's weights are conventionally identified by performing maximum likelihood estimation. Given a training data set Zn <sup>¼</sup> ð Þ y kð Þ; x kð Þ <sup>n</sup> k¼l , the task is to find a weight vector which minimizes the following cost function:

$$J\_n(w) = \frac{1}{n} \sum\_{k=1}^n \left( y(k) - \hat{y}(\mathbf{x}(k), w) \right)^2 \tag{18}$$

weights wi are computed as

inputs to the PCA.

where A<sup>P</sup>

n

<sup>w</sup><sup>i</sup> <sup>¼</sup> <sup>Y</sup><sup>n</sup> i¼1 μc l i

Figure 7. Data from navigation spaces in different zones and areas, (z1, z2, z3, …. zm, a1, a2, a3, …. An), they represent

� � <sup>δ</sup> <sup>ð</sup>y kð Þ <sup>k</sup>�nþ<sup>l</sup> is AP

inputs to the system, and u kð Þ¼ ðx1ð Þk , x2ð Þk , …, xnþ<sup>1</sup>ð ÞÞ k is the fuzzy system knowledge vec-

P<sup>n</sup> <sup>p</sup>¼<sup>1</sup> xp <sup>β</sup> � �vp

P<sup>n</sup>

p <sup>l</sup> yk <sup>þ</sup> …aP <sup>n</sup>yk�nþ<sup>l</sup> <sup>þ</sup> <sup>b</sup>Pu kð Þ � �

> p l

x kð Þ¼

tor. Typically, the output of the fuzzy decision based system is computed as:

n

� � <sup>δ</sup> ð Þ u kð Þ is B<sup>P</sup> � �

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

http://dx.doi.org/10.5772/intechopen.81195

79

� � and bP � � are constants, <sup>p</sup> <sup>¼</sup> ð Þ <sup>1</sup>; <sup>2</sup>;…:; <sup>n</sup>; u kð Þ are the

<sup>p</sup>¼<sup>1</sup> vp (23)

A dynamic TSK fuzzy system is constructed from the following rules:

l

) ynþ<sup>1</sup> <sup>¼</sup> <sup>a</sup>

if y k ð Þ ð Þ is A<sup>P</sup>

� � and B<sup>P</sup> � � are fuzzy sets, a

ð Þ xi (21)

(22)

As the knowledge based system, yxk ^ð Þ ð Þ; w is much interrelated with respect to the weights, linear optimization techniques cannot be applied. The adopted Neuro-fuzzy system has number of inputs (n) (representing the features) and (m) outputs (representing classes of features). In reference to Figure 7, there are dataset about the mobile area and zone of navigation. This is due to the large amount of information coming from the visual system. Here comes the potential of employing the PCA to reduce the dimensionality of the input spaces.

#### 4.3. Fuzzy decision based system

The last stage of the mobile robot maps learning system, is the fuzzy decision system. Within this stage, the hard decisions are undertaken by the mobile robot system during a course of navigation. A fuzzy system is constructed typically from the following rules:

$$\begin{aligned} D &= G \cap \mathbb{C} \Leftrightarrow \mu\_D(a) = \mu\_G(a) \bigwedge \mu\_{GC}(a), \text{ for } a \in A\\ a^\* &= \text{ARGmax}\{\mu\_G(a) \wedge \mu\_c(a)\}, \dots \quad a \in A \end{aligned} \tag{19}$$

The rules (if) parts, are identical to an ordinary fuzzy IF-THEN rules. Given an fuzzy inputs <sup>u</sup> <sup>¼</sup> ð Þ <sup>u</sup>1; <sup>u</sup>2;…:; …:un <sup>T</sup> the output y k ^ð Þ of a fuzzy system is computed as the weighted average of the yl <sup>s</sup>, that is:

$$\hat{y}(k) = \frac{\sum\_{l=l}^{m} y^l w^l}{\sum\_{l=l}^{m} w^l} \tag{20}$$

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces http://dx.doi.org/10.5772/intechopen.81195 79

Figure 7. Data from navigation spaces in different zones and areas, (z1, z2, z3, …. zm, a1, a2, a3, …. An), they represent inputs to the PCA.

weights wi are computed as

this layer, there is a circle node which multiplies incoming signals and sends their product out.

Yif <sup>i</sup> ! <sup>X</sup><sup>o</sup>

The consequent parameters thus identified are optimal (in the consequent parameter space) under the condition that the premise parameters are fixed. The knowledge system's weights are conventionally identified by performing maximum likelihood estimation. Given a training data set

linear optimization techniques cannot be applied. The adopted Neuro-fuzzy system has number of inputs (n) (representing the features) and (m) outputs (representing classes of features). In reference to Figure 7, there are dataset about the mobile area and zone of navigation. This is due to the large amount of information coming from the visual system. Here comes the

The last stage of the mobile robot maps learning system, is the fuzzy decision system. Within this stage, the hard decisions are undertaken by the mobile robot system during a course of

<sup>D</sup> <sup>¼</sup> <sup>G</sup> <sup>∩</sup><sup>C</sup> <sup>⇔</sup> <sup>μ</sup>Dð Þ¼ <sup>a</sup> <sup>μ</sup>Gð Þ<sup>a</sup> <sup>V</sup> <sup>μ</sup>GCð Þ<sup>a</sup> , for a <sup>∈</sup> <sup>A</sup>

The rules (if) parts, are identical to an ordinary fuzzy IF-THEN rules. Given an fuzzy inputs

P<sup>m</sup> <sup>i</sup>¼<sup>l</sup> yl wl

P<sup>m</sup>

<sup>i</sup> ¼ P <sup>i</sup> Yif P

, the task is to find a weight vector which minimizes the following cost function:

ð Þ y kð Þ� yxk ^ð Þ ð Þ; w

The output node computes the system output as summation of incoming signals, i.e.:

χ<sup>i</sup> ¼ μAix kð Þ� <sup>1</sup> μBiy kð Þ<sup>2</sup> for i ¼ 1, 2 (15)

i <sup>i</sup> Yi

th sample is obtained by the maximum f g Osk value as

<sup>2</sup> (18)

Cs ¼ maxk¼1, <sup>2</sup>,…:<sup>K</sup>f g Osk (17)

^ð Þ ð Þ; w is much interrelated with respect to the weights,

<sup>a</sup><sup>∗</sup> <sup>¼</sup> ARGmax <sup>μ</sup>Gð Þ<sup>a</sup> <sup>∧</sup> <sup>μ</sup>cð Þ<sup>a</sup> � �, …:: <sup>a</sup> <sup>∈</sup> <sup>A</sup> (19)

^ð Þ of a fuzzy system is computed as the weighted average

<sup>i</sup>¼<sup>l</sup> wl (20)

(16)

Stages of the adopted Neuro-fuzzy classifier, is shown in Figure 6.

Xo <sup>i</sup> <sup>¼</sup> <sup>X</sup> i

Jnð Þ¼ w

1 n Xn k¼1

potential of employing the PCA to reduce the dimensionality of the input spaces.

navigation. A fuzzy system is constructed typically from the following rules:

y k ^ð Þ¼

More precisely, the class label for the s

k¼l

As the knowledge based system, yxk

4.3. Fuzzy decision based system

<sup>u</sup> <sup>¼</sup> ð Þ <sup>u</sup>1; <sup>u</sup>2;…:; …:un <sup>T</sup> the output y k

<sup>s</sup>, that is:

of the yl

follows:

Zn <sup>¼</sup> ð Þ y kð Þ; x kð Þ <sup>n</sup>

78 Applications of Mobile Robots

$$w^i = \prod\_{i=1}^n \mu c\_i^l(\mathbf{x}\_i) \tag{21}$$

A dynamic TSK fuzzy system is constructed from the following rules:

$$\begin{aligned} \text{if } (\mathbf{y}(k)) \text{ is } \begin{pmatrix} A\_l^P \end{pmatrix} \delta \left( \mathbf{y}(k\_{k-n+l}) \text{ is } \begin{pmatrix} A\_n^P \end{pmatrix} \delta \left( \mathbf{u}(k) \right) \text{ is } \begin{pmatrix} B^P \end{pmatrix} \\\\ \Rightarrow \begin{pmatrix} \mathbf{y}\_{n+1} = a\_l^p y\_k + ... a\_n^p y\_{k-n+l} + b^p u(k) \end{pmatrix} \end{aligned} \tag{22}$$

where A<sup>P</sup> n � � and B<sup>P</sup> � � are fuzzy sets, a p l � � and bP � � are constants, <sup>p</sup> <sup>¼</sup> ð Þ <sup>1</sup>; <sup>2</sup>;…:; <sup>n</sup>; u kð Þ are the inputs to the system, and u kð Þ¼ ðx1ð Þk , x2ð Þk , …, xnþ<sup>1</sup>ð ÞÞ k is the fuzzy system knowledge vector. Typically, the output of the fuzzy decision based system is computed as:

$$\mathfrak{x}(k) = \frac{\sum\_{p=1}^{n} \mathfrak{x}^{p}(\boldsymbol{\beta}) \boldsymbol{\upsilon}^{p}}{\sum\_{p=1}^{n} \mathfrak{v}^{p}} \tag{23}$$


Table 1. Fuzzy decision based system input-outputs representation.


Table 2 Neuro-fuzzy classifier, input-outputs representation.

where xpð Þ <sup>k</sup><sup>1</sup> is given in Eq. (23) and:

$$\upsilon^p = \prod\_{i=1}^n \mu A\_i^p(\mathbf{x}\_k) \mu B^p(\boldsymbol{\mu}(k)) \tag{24}$$

5.1. Behaviour knowledge building

Figure 8. Implementation system hierarchy.

5.2. Navigation intelligence

The classifier inputs:

The classifier outputs:

For building the mobile robot behaviour at localities, the mobile system was maneuvered over a space in the laboratory for several trails. Typical physical readings from the robot odometry, and sensory observations were recorded. Typical readings are shown in Figure 9. Different mobile behaviors (for learning) were also recorded, beside the odometry, and sensory observations.

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

http://dx.doi.org/10.5772/intechopen.81195

81

Building the mobile robot navigation intelligence is the next phase. This phase requires blinding all the previous inputs (readings, situations, and behaviors). This will help to take the most appropriate actions. The designated learning and decision making architecture is a

Neuro-fuzzy. Typical information, that constitute the Neuro-fuzzy classifier inputs are:

ZONES of navigation. This represents typical zones where the mobile robot is located. AREAS of navigation. This represents typical areas where the mobile robot is moving.

SEGMENT of navigation. This represents typical segment where the mobile robot is moving. OBSERVATIONS. This represents typical Observations, the mobile is experiencing at a locality.

The mobile robot training datasets, do consist of four inputs. There are also four output parameters. This is summarized in Tables 1 and 2.

#### 5. The experimentation

Within this section, we shall discuss few experimentations results. In order to implement the proposed navigation methodology, the (914 PC BOT) has been reengineered in such a way to allow more control and observations to be communicated through. The main high-level coding was achieved using Matlab. Matlab toolboxes have been integrated in such a way to allow PCA computation, Neuro-fuzzy learning capabilities, and fuzzy decision making routines. This is further indicated to in Figure 8.

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces http://dx.doi.org/10.5772/intechopen.81195 81

Figure 8. Implementation system hierarchy.

#### 5.1. Behaviour knowledge building

For building the mobile robot behaviour at localities, the mobile system was maneuvered over a space in the laboratory for several trails. Typical physical readings from the robot odometry, and sensory observations were recorded. Typical readings are shown in Figure 9. Different mobile behaviors (for learning) were also recorded, beside the odometry, and sensory observations.

#### 5.2. Navigation intelligence

where xpð Þ <sup>k</sup><sup>1</sup> is given in Eq. (23) and:

Identification of zone of navigation

80 Applications of Mobile Robots

Identification of area of locality

Identification of segment of navigation

Mobile robot delicate sensory observation

1 First mobile stimuli

2 Second mobile stimuli

3 Third mobile stimuli

4 Fourth mobile stimuli

5. The experimentation

This is further indicated to in Figure 8.

parameters. This is summarized in Tables 1 and 2.

Table 2 Neuro-fuzzy classifier, input-outputs representation.

Table 1. Fuzzy decision based system input-outputs representation.

area, ..

Inputs Outputs

Zones identification, and obstacles in zones, z1, z2, z3, z4, z5, ……. zm

Areas identification, and obstacles in areas. Obstacles in a1: area floor, obstacles in a2: Out\_Corridor, Obstacles in a3: building. Entry, Obstacles in a4:

Obstacles at different segments within an

Rotate around, move robot forward, move robot backward, rotate right, rotate left, …

> vp <sup>¼</sup> <sup>Y</sup><sup>n</sup> i¼1

μA<sup>p</sup>

Inputs Outputs x Robot Zone O1 Behavior1 y Robot Area O2 Behavior2 z Robot Segment O3 Behavior3 w Delicate Observations O4 Behavior4

The mobile robot training datasets, do consist of four inputs. There are also four output

Within this section, we shall discuss few experimentations results. In order to implement the proposed navigation methodology, the (914 PC BOT) has been reengineered in such a way to allow more control and observations to be communicated through. The main high-level coding was achieved using Matlab. Matlab toolboxes have been integrated in such a way to allow PCA computation, Neuro-fuzzy learning capabilities, and fuzzy decision making routines.

<sup>i</sup> ð Þ xk <sup>μ</sup>Bp

ð Þ u kð Þ (24)

Rotate around, move robot forward, move robot backward, rotate right, rotate

Image focus, image capture, … image processing of a scene.

Video recording zooming with

video capture, ..

Delicate mobile action.

left, …

First mobile behavior. Behavior1

Second mobile behaviour. Behavior2

Fourth mobile behaviour. Behavior4

Third mobile behaviour. Behavior3

> Building the mobile robot navigation intelligence is the next phase. This phase requires blinding all the previous inputs (readings, situations, and behaviors). This will help to take the most appropriate actions. The designated learning and decision making architecture is a Neuro-fuzzy. Typical information, that constitute the Neuro-fuzzy classifier inputs are:

The classifier inputs:

ZONES of navigation. This represents typical zones where the mobile robot is located.

AREAS of navigation. This represents typical areas where the mobile robot is moving.

SEGMENT of navigation. This represents typical segment where the mobile robot is moving.

OBSERVATIONS. This represents typical Observations, the mobile is experiencing at a locality.

The classifier outputs:

If (Mobile is in zone1 ….. and in area1 ….) then (do image FOUCS).

for mobile robot maps learning with navigating capabilities.

Address all correspondence to: ebmattar@uob.edu.bh

Robotic Systems. 2004;1(1):15-23 ISSN 1729-8806

Engineering; 1997. pp. 126-132

College of Engineering, University of Bahrain, Sukhair, Kingdom of Bahrain

[1] Janglová D. Neural networks in mobile robot motion. Inernational Journal of Advanced

[2] Raymond A, Tayib S, Hall L. Remote controlled, vision guided, mobile robot system. In: Intelligent Robots and Computer Vision XVI: Algorithms, Techniques, Active Vision, and Materials Handling, vol. 3208; Proceedings of SPIE. The International Society for Optical

navigation and behaviors tasks.

6. Conclusions

Author details

Ebrahim A. Mattar

References

If (Mobile in zone41 and in area2 and, … and Special task, then (move back).

If (Mobile is in zone1, ….. and in segment2 and, Mobile special task) then (set an ALAM and GAZE).

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

http://dx.doi.org/10.5772/intechopen.81195

83

Given the defined navigation strategy, the mobile robot is able to undertake much detailed

Learning mobile robot navigation behavior, is an essential feature, for improved navigation. In addition, it helps to gain further understanding about the spaces of navigation. In this study, navigation maps details have been created while relying on dataset collected by SLAM routines. Due to enormous sensory and environmental data observation to be analyzed during navigation, we have reduced the dimensionally and size of environmental and sensory observation information with PCA technique. Reduction of environment information (i.e. getting features), are hence used as learning inputs to a neuro-fuzzy classifier. Examples of Neurofuzzy feature inputs are, navigation locations, areas, …, and behaviors related to particular localities. The final stage of mobile robot map building is a fuzzy decision based system. Within this stage, mobile robot navigation decisions are undertaken. With multi-levels of mobile robot sensory and navigation observation dataset, we have designed a learning system

If (Mobile is in zone3 and in area5 and segment 3, and Image Capture), … then (do image analyze).

Figure 9. Mobile robot real sensory observations, by experimentation. Dataset have been collected through a number of runs for the PCA.

First mobile behavior. Behavior1, Rotate Around, Move Robot forward, Move Robot Backward, Rotate Right, Rotate Left, ….

Second mobile behavior. Behavior2, Image Processing of a Scene, … .

Third mobile behavior. Behavior3, Video Recording.

Fourth mobile behavior. Behavior4, Delicate Mobile Action.

With such inputs and system outputs, a good degree of a mixture of mobile behaviors can therefore be created. This is further listed below:

The implementation system hierarchy, is shown in Figure 8. An adequate of mobile intelligence was created for a mobile navigation within hazardous environments. Inputs to the Neuro-fuzzy decision based system are coming from the PCA network.

#### 5.3. Fuzzy if-then decision system

In addition, the four system inputs-outputs, do represent the system outputs the mobile robot should undertake also at any particular situation. While relying on the fuzzy (if-then) statements, we are able to make further final decision to be undertaken by the mobile robot. Within this sense, we are able to build an (if then statement), as follows:

Typical Fuzzy Rules are:

If (Input\_#1 is ….. and Input\_#2 is ….) then (Output\_#1 is …. and Output\_2 is ….) .. ...

If (Input\_#3 is ….. and Input\_#2 is ….) then (Output\_#4 is …. and Output\_2 is ….) .. ...

If (Mobile is in zone1 ….. and in area1 ….) then (do image FOUCS).

If (Mobile is in zone1, ….. and in segment2 and, Mobile special task) then (set an ALAM and GAZE).

If (Mobile is in zone3 and in area5 and segment 3, and Image Capture), … then (do image analyze).

If (Mobile in zone41 and in area2 and, … and Special task, then (move back).

Given the defined navigation strategy, the mobile robot is able to undertake much detailed navigation and behaviors tasks.

#### 6. Conclusions

Learning mobile robot navigation behavior, is an essential feature, for improved navigation. In addition, it helps to gain further understanding about the spaces of navigation. In this study, navigation maps details have been created while relying on dataset collected by SLAM routines. Due to enormous sensory and environmental data observation to be analyzed during navigation, we have reduced the dimensionally and size of environmental and sensory observation information with PCA technique. Reduction of environment information (i.e. getting features), are hence used as learning inputs to a neuro-fuzzy classifier. Examples of Neurofuzzy feature inputs are, navigation locations, areas, …, and behaviors related to particular localities. The final stage of mobile robot map building is a fuzzy decision based system. Within this stage, mobile robot navigation decisions are undertaken. With multi-levels of mobile robot sensory and navigation observation dataset, we have designed a learning system for mobile robot maps learning with navigating capabilities.

#### Author details

First mobile behavior. Behavior1, Rotate Around, Move Robot forward, Move Robot Back-

Figure 9. Mobile robot real sensory observations, by experimentation. Dataset have been collected through a number of

With such inputs and system outputs, a good degree of a mixture of mobile behaviors can

The implementation system hierarchy, is shown in Figure 8. An adequate of mobile intelligence was created for a mobile navigation within hazardous environments. Inputs to the

In addition, the four system inputs-outputs, do represent the system outputs the mobile robot should undertake also at any particular situation. While relying on the fuzzy (if-then) statements, we are able to make further final decision to be undertaken by the mobile robot. Within

If (Input\_#1 is ….. and Input\_#2 is ….) then (Output\_#1 is …. and Output\_2 is ….) .. ... If (Input\_#3 is ….. and Input\_#2 is ….) then (Output\_#4 is …. and Output\_2 is ….) .. ...

Second mobile behavior. Behavior2, Image Processing of a Scene, … .

Neuro-fuzzy decision based system are coming from the PCA network.

this sense, we are able to build an (if then statement), as follows:

ward, Rotate Right, Rotate Left, ….

runs for the PCA.

82 Applications of Mobile Robots

5.3. Fuzzy if-then decision system

Typical Fuzzy Rules are:

Third mobile behavior. Behavior3, Video Recording.

therefore be created. This is further listed below:

Fourth mobile behavior. Behavior4, Delicate Mobile Action.

Ebrahim A. Mattar

Address all correspondence to: ebmattar@uob.edu.bh

College of Engineering, University of Bahrain, Sukhair, Kingdom of Bahrain

#### References


[3] Gang P, Xinhan H, Jian G, Cheng L. Camera modeling and distortion calibration for mobile robot vision. In: Proceedings of the 7th World Congress on Intelligent Control and Automation (WCICA), WCICA'08. 2008. pp. 1657-1662

[17] Abdul B, Robert S, Jason G, Yahya K, Usman M, Muhammad H, et al. Stereo vision based self-localization of autonomous mobile robots. In: Proceedings (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Robot Vision—

Mobile Robot Feature-Based SLAM Behavior Learning, and Navigation in Complex Spaces

http://dx.doi.org/10.5772/intechopen.81195

85

[18] Hall L, Liao X, Alhaj AM. Learning for intelligent mobile robots. In: Proceedings of SPIE,

[19] Ya-Chun C, Hidemasa K, Yoshio Y. Novel application of a laser range finder with vision system for wheeled mobile robot. In: Proceedings of the IEEE/ASME International Con-

[20] Nazari V, Naraghi M. A vision-based intelligent path following control of a four-wheel differentially driven skid steer mobile robot. In: Proceedings of the 10th International Conference on Control, Automation, Robotics and Vision, ICARCV-2008. 2008. pp. 378-383 [21] Mohan S, Peter S. Color learning and illumination invariance on mobile robots: A survey.

[22] Masayuki I, Fumio K, Satoshi K, Hirochika I. Vision-equipped apelike robot based on the remote-brained approach. In: Proceedings IEEE International Conference on Robotics and

[23] Xiaoning C, Yuqing H, Gang L, Jin G. Localization algorithm of mobile robot based on single vision and laser radar. In: Proceedings of the 7th World Congress on Intelligent

[24] Andreja K, Sanjin B, Ivan P. Mobile robot self-localization in complex indoor environments using monocular vision and 3D model. In: IEEE/ASME International Conference on

[25] Takeshi S, Hashimoto H. Human observation based mobile robot navigation in intelligent space. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS

[26] Manoj K, Ernest L H. Intelligent robot control using omnidirectional vision. In: Proceedings of SPIE, vol. 1825. The International Society for Optical Engineering; 1993. pp. 573-584

[27] Robot Cartograsphy: ROS + SLAM. http://www.pirobot.org/blog/0015/

Second International Workshop, RobVis., vol. 493-1 LNCS. 2008. pp. 367-380

vol. 5267. The International Society for Optical Engineering; 2003. pp. 12-25

ference on Advanced Intelligent Mechatronics, AIM. 2008. pp. 280-285

Robotics and Autonomous Systems. 2009;57(6–7):629-644

Control and Automation, WCICA'08. 2008. pp. 7661-7666

Automation, vol. 2. 1995. pp. 2193-2198

Advanced Intelligent Mechatronics AIM. 2007

2006. 2006. pp. 1044-1049


[17] Abdul B, Robert S, Jason G, Yahya K, Usman M, Muhammad H, et al. Stereo vision based self-localization of autonomous mobile robots. In: Proceedings (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Robot Vision— Second International Workshop, RobVis., vol. 493-1 LNCS. 2008. pp. 367-380

[3] Gang P, Xinhan H, Jian G, Cheng L. Camera modeling and distortion calibration for mobile robot vision. In: Proceedings of the 7th World Congress on Intelligent Control

[4] Bonin-Font F, Ortiz A, Oliver G. Visual navigation for mobile robots: A survey. Journal of

[5] Abdul B, Robert S, Yahya K, Gloom M. A hybrid approach towards vision based selflocalization of autonomous mobile robots. In: Proceedings of the International Conference

[6] Araujo R. Prune-able fuzzy ART neural architecture for robot map learning and naviga-

[7] Vlassis N, Papakonstantinou G, Tsanakas P. Robot map building by Kohonen's selforganizing neural networks. In: Proceedings of 1st Mobinet Symposium; Athens, Greece.

[8] Filliata D, Meyer J. Map-based navigation in mobile robots: I. A review of localization

[9] Miftahur R, Rahman M, Rizvi H, Haque Abul L, Towhidul IM. Architecture of the vision system of a line following mobile robot operating in static environment. In: Pakistan

[10] Tomohiro S, Yoshio M, Taichi K, Masayuki I, Hirochika I. Development and integration of generic components for a teachable vision-based mobile robot. IEEE/ASME Transactions

[11] Ryosuke M, Fumiaki T, Fumio M. Skill acquisition of a ball lifting task using a mobile robot with a monocular vision system. In: IEEE International Conference on Intelligent

[12] Vlassis N, Papakonstantinou G, Tsanakas P. Robot map building by Kohonen's selforganizing neural networks. In: Proceedings of the 1st Mobinet Symposium; Greece. 1997

[13] Changhan P, Sooin K, Joonki P. Stereo vision-based autonomous mobile robot. In: Intelligent Robots and Computer Vision XXIII: Algorithms, Techniques, and Active Vision; vol.

[14] Thrun S. Learning metric-topological maps for indoor mobile robot navigation. Artificial

[15] Nima F, Mohammad T, Sadaf S. Intelligent real time control of mobile robot based on

[16] Hall EL, Ghaffari M, Liao X, Alhaj ASM. Intelligent robot control using an adaptive critic with a task control center and dynamic database. In: Intelligent Robots and Computer Vision XXIV: Algorithms, Techniques, and Active Vision, vol. 6384. Proceedings of SPIE.

image processing. IEEE Proceedings Intelligent Vehicles. 2007;IV:410-415

The International Society for Optical Engineering; 2006

6/6. Proceedings of SPIE. The International Society for Optical Engineering; 2005

tion in dynamic environments. IEEE Neural Network Transactions. 2006;17(4)

and Automation (WCICA), WCICA'08. 2008. pp. 1657-1662

on Machine Vision, ICMV-2007. 2007. pp. 1-6

strategies. Cognitive Systems Research. 2003;4:243-282

Section Multitopic Conference (INMIC 2005). 2005

on Mechatronics. 1996;1(3):230-236

Intelligence Journal. 1998;99:21-71

Robots and Systems. 2006. pp. 5141-5146

1997

84 Applications of Mobile Robots

Intell Robot System. 2008;53:263-296. DOI: 10.1007/s10846-008-9235-4


**Chapter 5**

Provisional chapter

**Mobile Robot Navigation in Indoor Environments:**

DOI: 10.5772/intechopen.79842

Mobile Robot Navigation in Indoor Environments:

**Geometric, Topological, and Semantic Navigation**

The objective of the chapter is to show current trends in robot navigation systems related to indoor environments. Navigation systems depend on the level of abstraction of the environment representation. The three main techniques for representing the environment will be described: geometric, topological, and semantic. The geometric representation of the environment is closer to the sensor and actuator world and it is the best one to perform local navigation. Topological representation of the environment uses graphs to model the environment and it is used in large navigation tasks. The semantic representation is the most abstract representation model and adds concepts such as utilities or meanings of the environment elements in the map representation. In addition, regardless of the representation used for navigation, perception plays a signif-

icant role in terms of understanding and moving through the environment.

Keywords: robot navigation, environment modeling, topological navigation, semantic

Navigation of mobile robots has been traditionally understood as solving the problem pro-

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

Geometric, Topological, and Semantic Navigation

Ramón Barber, Jonathan Crespo, Clara Gómez,

Ramón Barber, Jonathan Crespo, Clara Gómez, Alejandra C. Hernámdez and Marina Galli

Alejandra C. Hernámdez and Marina Galli

http://dx.doi.org/10.5772/intechopen.79842

navigation, geometric navigation

1. Introduction: navigation problem

posed by these three questions (Levitt [1]):

• Where are other places related to me? • How do I get to other places from here?

• Where am I?

Abstract

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

#### **Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation** Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

DOI: 10.5772/intechopen.79842

Ramón Barber, Jonathan Crespo, Clara Gómez, Alejandra C. Hernámdez and Marina Galli Ramón Barber, Jonathan Crespo, Clara Gómez, Alejandra C. Hernámdez and Marina Galli

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79842

#### Abstract

The objective of the chapter is to show current trends in robot navigation systems related to indoor environments. Navigation systems depend on the level of abstraction of the environment representation. The three main techniques for representing the environment will be described: geometric, topological, and semantic. The geometric representation of the environment is closer to the sensor and actuator world and it is the best one to perform local navigation. Topological representation of the environment uses graphs to model the environment and it is used in large navigation tasks. The semantic representation is the most abstract representation model and adds concepts such as utilities or meanings of the environment elements in the map representation. In addition, regardless of the representation used for navigation, perception plays a significant role in terms of understanding and moving through the environment.

Keywords: robot navigation, environment modeling, topological navigation, semantic navigation, geometric navigation

#### 1. Introduction: navigation problem

Navigation of mobile robots has been traditionally understood as solving the problem proposed by these three questions (Levitt [1]):


© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

The answer to these questions refers to localization to determine where the robot is, pathplanning to know how to reach other places from where the robot is, and navigation to perform the trajectory commanded by the path-planning system.

landmarks. These authors tackled for the first time robot navigation concepts from a topological approach and some of them noticed the importance of perception in the navigational

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

89

At the last level of the semantic navigation paradigm, the ability to reason and to infer new knowledge is required. In today's world of robotics, there is a general tendency to implement behavioral mechanisms based on human psychology, taking as example the natural processing of thought. This allows a greater understanding of the environment and the objects it contains. This trend is also valued in the field of mobile robot navigation. Navigators have been increasing their level of abstraction over time. Initially, navigation was solved with geometric navigators that interpreted the environment as a set of accessible areas and areas that were not accessible. Then, the concept of node and the accessibility between nodes was introduced, which allowed increasing the level of abstraction generating graphs and calculating trajectories with algorithms of graphs. However, the level of abstraction has increased a step further, introducing high-level concepts that classify the rooms according to more complex and

Another important aspect is the collection of the information of the environment, which has to be available for the navigation systems, among other tasks carried out by mobile robots. This information is provided by a perceptual system, therefore non-trivial problems appear related to object detection and place recognition. One of the most challenging issues in scene recognition is the appearance of a place. Sometimes, the same place may look different or different places may look similar. Also, the position of the robot and the variation of its point of view can affect the identification of the place when it is revisited. Environment models and the way to define the navigation tasks can be remodeled taking into account new technologies and trends. This will provide more autonomy to mobile robots and will help the interaction with humans in their usual environments. In this trend, vision is the main sensor for navigation, localization, and scene recognition. Place recognition is a very well-known challenging problem that not only has to do with how a robot can give the same meaning that a human do to the same image, but also with the variability in the appearance of these images in the real world. Place recognition has a strong relation with many major robotics research fields includ-

Geometric navigation consists of moving the robot from one point of the environment to another one, given those points by its coordinates in a map. The map of the environment is classically represented by a grid of points, and the trajectory between two points is a sequence of points the robot must reach in the given order. The controller of the robot must reach the next point of the path closing a loop with the encoder information about its position (distance and angle). One of the main objectives in the geometric navigation researches is the pathplanning task from an initial point to a final point, creating an algorithm able to find a path

abstract data such as the utility and the objects they contain.

ing simultaneous localization and mapping.

2. Geometric navigation

ensuring completeness.

processes.

This approach has ruled robot navigation paradigm and many successful solutions have been proposed. However, the advances in technical developments allow a wider thought of mobile robot navigation giving a solution for a "bigger" problem. This solution would give answer to two additional questions, which are:


The answer to these new questions focuses on the importance of perception to determine the place and the elements of the place where the robot is and the importance of modeling the environment to determine the structure of the place and infer the connections between places. When robots are integrated in human environments and live together with humans, perception and modeling gain importance to enable future tasks related to service robots such as manipulation or human-robot interaction. In this chapter, we give a conceptual answer to the traditional robot navigation question and to the questions proposed above.

Robot navigation has been tackled from different perspectives leading to a classification into three main approaches: geometric navigation, topological navigation, and semantic navigation. Although the three of them differ in their definition and methods, all of them have the focus of answering the same questions.

From the beginning, authors have focused on generating metric maps and moving through the map using metric path planners. The most well-known algorithm to build metric maps is simultaneous localization and mapping (SLAM) as proposed by Bailey & Durrant-Whyte [2]. Wang et al. [3] use SLAM and Rapidly-exploring Random Tree (RTT) planning with Monte Carlo localization to drive a wheelchair in indoor environments. Pfrunder et al. [4] use SLAM and occupancy grids to navigate in heterogeneous environments. However, as larger maps are considered, it is computationally expensive to keep metric maps and other authors have focused on topological representations. Fernández-Madrigal et al. [5], design a hierarchical topological model to drive a wheelchair and perform reactive navigation and path-planned navigation. Ko et al. [6] works with topological maps where nodes are bags of visual words and a Bayesian framework is used for localization.

Regarding topological navigation, since the first developments, the global conception of the system has attracted the interest of several authors. In Kuipers & Levitt [7], the authors presented a four-level hierarchy (sensorimotor interaction, procedural behaviors, topological mapping, and metric mapping) to plan trajectories and execute them. Giralt et al. [8] defined a navigation and control system integrating modeling, planning, and motion control and stated that the key to autonomous robot was system integration and multisensory-driven navigation. Mataric [9] defined a distributed navigation model whose main purposes were collision-free path-planning, landmark detection, and environment learning. In Levitt [1], Levitt established a visual-based navigation where landmarks are memorized and paths are sequences of landmarks. These authors tackled for the first time robot navigation concepts from a topological approach and some of them noticed the importance of perception in the navigational processes.

At the last level of the semantic navigation paradigm, the ability to reason and to infer new knowledge is required. In today's world of robotics, there is a general tendency to implement behavioral mechanisms based on human psychology, taking as example the natural processing of thought. This allows a greater understanding of the environment and the objects it contains. This trend is also valued in the field of mobile robot navigation. Navigators have been increasing their level of abstraction over time. Initially, navigation was solved with geometric navigators that interpreted the environment as a set of accessible areas and areas that were not accessible. Then, the concept of node and the accessibility between nodes was introduced, which allowed increasing the level of abstraction generating graphs and calculating trajectories with algorithms of graphs. However, the level of abstraction has increased a step further, introducing high-level concepts that classify the rooms according to more complex and abstract data such as the utility and the objects they contain.

Another important aspect is the collection of the information of the environment, which has to be available for the navigation systems, among other tasks carried out by mobile robots. This information is provided by a perceptual system, therefore non-trivial problems appear related to object detection and place recognition. One of the most challenging issues in scene recognition is the appearance of a place. Sometimes, the same place may look different or different places may look similar. Also, the position of the robot and the variation of its point of view can affect the identification of the place when it is revisited. Environment models and the way to define the navigation tasks can be remodeled taking into account new technologies and trends. This will provide more autonomy to mobile robots and will help the interaction with humans in their usual environments. In this trend, vision is the main sensor for navigation, localization, and scene recognition. Place recognition is a very well-known challenging problem that not only has to do with how a robot can give the same meaning that a human do to the same image, but also with the variability in the appearance of these images in the real world. Place recognition has a strong relation with many major robotics research fields including simultaneous localization and mapping.

#### 2. Geometric navigation

The answer to these questions refers to localization to determine where the robot is, pathplanning to know how to reach other places from where the robot is, and navigation to

This approach has ruled robot navigation paradigm and many successful solutions have been proposed. However, the advances in technical developments allow a wider thought of mobile robot navigation giving a solution for a "bigger" problem. This solution would give answer to

The answer to these new questions focuses on the importance of perception to determine the place and the elements of the place where the robot is and the importance of modeling the environment to determine the structure of the place and infer the connections between places. When robots are integrated in human environments and live together with humans, perception and modeling gain importance to enable future tasks related to service robots such as manipulation or human-robot interaction. In this chapter, we give a conceptual answer to the

Robot navigation has been tackled from different perspectives leading to a classification into three main approaches: geometric navigation, topological navigation, and semantic navigation. Although the three of them differ in their definition and methods, all of them have the

From the beginning, authors have focused on generating metric maps and moving through the map using metric path planners. The most well-known algorithm to build metric maps is simultaneous localization and mapping (SLAM) as proposed by Bailey & Durrant-Whyte [2]. Wang et al. [3] use SLAM and Rapidly-exploring Random Tree (RTT) planning with Monte Carlo localization to drive a wheelchair in indoor environments. Pfrunder et al. [4] use SLAM and occupancy grids to navigate in heterogeneous environments. However, as larger maps are considered, it is computationally expensive to keep metric maps and other authors have focused on topological representations. Fernández-Madrigal et al. [5], design a hierarchical topological model to drive a wheelchair and perform reactive navigation and path-planned navigation. Ko et al. [6] works with topological maps where nodes are bags of visual words

Regarding topological navigation, since the first developments, the global conception of the system has attracted the interest of several authors. In Kuipers & Levitt [7], the authors presented a four-level hierarchy (sensorimotor interaction, procedural behaviors, topological mapping, and metric mapping) to plan trajectories and execute them. Giralt et al. [8] defined a navigation and control system integrating modeling, planning, and motion control and stated that the key to autonomous robot was system integration and multisensory-driven navigation. Mataric [9] defined a distributed navigation model whose main purposes were collision-free path-planning, landmark detection, and environment learning. In Levitt [1], Levitt established a visual-based navigation where landmarks are memorized and paths are sequences of

traditional robot navigation question and to the questions proposed above.

perform the trajectory commanded by the path-planning system.

two additional questions, which are:

focus of answering the same questions.

and a Bayesian framework is used for localization.

• How is the structure of the environment I am in?

• How is this place like?

88 Applications of Mobile Robots

Geometric navigation consists of moving the robot from one point of the environment to another one, given those points by its coordinates in a map. The map of the environment is classically represented by a grid of points, and the trajectory between two points is a sequence of points the robot must reach in the given order. The controller of the robot must reach the next point of the path closing a loop with the encoder information about its position (distance and angle). One of the main objectives in the geometric navigation researches is the pathplanning task from an initial point to a final point, creating an algorithm able to find a path ensuring completeness.

Although the path-planning task is widely treated in mobile robots, computational capacity has increased exponentially and more complex algorithms can be developed. Algorithms try to find the shortest or fastest path while maintaining safety constraints. Another challenge is to try to get solutions that provide smoother trajectories, trying to imitate the human trajectories.

On the one hand, the attractive potential must be a function of the distance to the final destination, decreasing when the robot approaches this point. On the other hand, the repulsive potential should only be influenced when the robot is at a dangerous distance from obstacles. Therefore, the potential fields method allows to be performed in real time, as a local planner,

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

91

The problem that exists in this type of method is the local minimums [13], places where the

The generation of Voronoi diagrams seeks to maximize the distance between the robot and the obstacles, looking for the safest path between two points in the space [14]. In this way, the diagram is defined as the locus of the configurations that are at the same distance from

The algorithm divides the space into sections formed by vertices and segments that fulfill the cost function of maximum distance between obstacles, Figure 1. Then, the trajectory is sought from an initial point to the objective. For a more realistic representation, obstacles are consid-

One way of building the Voronoi diagram is using image-processing methods (skeletonization) based on the method of Breu [15]. These present a linear (and therefore asymptotically optimal)

The algorithm is built with the pixels that result after performing the morphological operations on the image, with the nodes being the points where the lines that pass through the pixels of

A trajectory generated by Voronoi diagrams has the disadvantage that it is not optimal from the point of view of length traveled, it can also present a large number of turns [16]. Moreover,

Over the years, a large number of solutions for the navigation problem have been presented using algorithms with random components, specifically in generic environments and with a

Probabilistic roadmap (PRM) is a trajectory planner that searches the connectivity of different points in the free space from a starting point to the final goal avoiding collisions with obstacles,

If the environment in which the robot is located is very complex, this type of algorithm allows finding a solution without a large computational requirement, so it is used in environments

considering obstacles only when the robot is at a minimum distance from them.

potential is null but it is not the final position.

ered as polygons, since physically an obstacle is not a point.

this method is not efficient for more than two dimensions.

time algorithm in order to calculate the Euclidean distance of a binary image.

2.1.2. Voronoi diagram

the obstacles.

the image intersect.

2.2. Probabilistic algorithms

large number of dimensions.

using random sampling methods [17].

with a large number of dimensions.

2.2.1. PRM algorithm

In the literature, many different algorithms can be found. One of the most important precedents were the works of LaValle [10], where a classification into two big groups depending on the way information is discretized is proposed: combinatorial planning and samplingbased planning. Combinatorial planning constructs structures containing all the necessary information for route planning; see De Berg et al. [11]. Sampling-based planning is based on an incremental representation of space and uses collision-free algorithms for path search. Here are some of the algorithms most used in the path-finding problem.

#### 2.1. Deterministic algorithms

One of the first approaches tries to get all the possible paths between two points and choose the shortest one. Two example algorithms use potential fields and Voronoi diagrams.

#### 2.1.1. Potential fields

The potential fields method is based on reactive planning techniques, which can be used to plan locally in unexplored environments. This method assigns to the obstacles similar characteristics that an electrostatic potential might have; in this way, the robot is considered as a particle under the influence of an artificial potential field that pulls it toward a target position, Figure 1a. The generation of trajectories is due to an attractive field toward the final position and another repulsive one with respect to the obstacles [12].

In this way, navigation through potential fields is composed of three phases:


Figure 1. Trajectories created using potential fields and Voronoi diagram methods. (a) Trajectory of a real robot created using Potential Fields Method, (b) Trajectories created using Voronoi Diagram by Skeletonization and (c) Trajectories created by Voronoi Diagram Method.

On the one hand, the attractive potential must be a function of the distance to the final destination, decreasing when the robot approaches this point. On the other hand, the repulsive potential should only be influenced when the robot is at a dangerous distance from obstacles. Therefore, the potential fields method allows to be performed in real time, as a local planner, considering obstacles only when the robot is at a minimum distance from them.

The problem that exists in this type of method is the local minimums [13], places where the potential is null but it is not the final position.

#### 2.1.2. Voronoi diagram

Although the path-planning task is widely treated in mobile robots, computational capacity has increased exponentially and more complex algorithms can be developed. Algorithms try to find the shortest or fastest path while maintaining safety constraints. Another challenge is to try to get solutions that provide smoother trajectories, trying to imitate the human trajectories. In the literature, many different algorithms can be found. One of the most important precedents were the works of LaValle [10], where a classification into two big groups depending on the way information is discretized is proposed: combinatorial planning and samplingbased planning. Combinatorial planning constructs structures containing all the necessary information for route planning; see De Berg et al. [11]. Sampling-based planning is based on an incremental representation of space and uses collision-free algorithms for path search. Here

One of the first approaches tries to get all the possible paths between two points and choose

The potential fields method is based on reactive planning techniques, which can be used to plan locally in unexplored environments. This method assigns to the obstacles similar characteristics that an electrostatic potential might have; in this way, the robot is considered as a particle under the influence of an artificial potential field that pulls it toward a target position, Figure 1a. The generation of trajectories is due to an attractive field toward the final position

Figure 1. Trajectories created using potential fields and Voronoi diagram methods. (a) Trajectory of a real robot created using Potential Fields Method, (b) Trajectories created using Voronoi Diagram by Skeletonization and (c) Trajectories

the shortest one. Two example algorithms use potential fields and Voronoi diagrams.

are some of the algorithms most used in the path-finding problem.

and another repulsive one with respect to the obstacles [12].

• Generation of the movement orders of the robot.

In this way, navigation through potential fields is composed of three phases:

• Calculation of the potential acting on the robot using the sensor data. • Determination of the vector of the artificial force acting on the robot.

2.1. Deterministic algorithms

created by Voronoi Diagram Method.

2.1.1. Potential fields

90 Applications of Mobile Robots

The generation of Voronoi diagrams seeks to maximize the distance between the robot and the obstacles, looking for the safest path between two points in the space [14]. In this way, the diagram is defined as the locus of the configurations that are at the same distance from the obstacles.

The algorithm divides the space into sections formed by vertices and segments that fulfill the cost function of maximum distance between obstacles, Figure 1. Then, the trajectory is sought from an initial point to the objective. For a more realistic representation, obstacles are considered as polygons, since physically an obstacle is not a point.

One way of building the Voronoi diagram is using image-processing methods (skeletonization) based on the method of Breu [15]. These present a linear (and therefore asymptotically optimal) time algorithm in order to calculate the Euclidean distance of a binary image.

The algorithm is built with the pixels that result after performing the morphological operations on the image, with the nodes being the points where the lines that pass through the pixels of the image intersect.

A trajectory generated by Voronoi diagrams has the disadvantage that it is not optimal from the point of view of length traveled, it can also present a large number of turns [16]. Moreover, this method is not efficient for more than two dimensions.

#### 2.2. Probabilistic algorithms

Over the years, a large number of solutions for the navigation problem have been presented using algorithms with random components, specifically in generic environments and with a large number of dimensions.

#### 2.2.1. PRM algorithm

Probabilistic roadmap (PRM) is a trajectory planner that searches the connectivity of different points in the free space from a starting point to the final goal avoiding collisions with obstacles, using random sampling methods [17].

If the environment in which the robot is located is very complex, this type of algorithm allows finding a solution without a large computational requirement, so it is used in environments with a large number of dimensions.

One of the main problems of PRM is that the solutions it finds do not have to be the optimal trajectory. Also, since the way it generates the nodes is completely random, it produces a nonhomogeneous distribution of the samples in the space. These two disadvantages are seen in Figure 2a.

#### 2.2.2. Rapidly exploring random tree algorithm

Rapidly exploring random tree (RRT) [18] provides a solution by creating random branches from a starting point. The collision-free branches are stored iteratively and new ones are created until the target point is reached. The algorithm is started with a tree whose source is a single node, the starting point. In each iteration, the tree expands by selecting a random state and expanding the tree to that state. The expansion is performed by extending the nearest node of the tree to the selected random state, which will depend on the size of the selected step. The algorithm creates branches until the tree takes a certain extension approaching the goal.

The size of the step is an important parameter of the algorithm. Small values result in slow expansion, but with finer paths or paths that can take fine turns [19].

discretization of the environment in a grid of cells, on which the Eikonal equation is solved

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

93

The trajectories obtained using FMM present two drawbacks: great abruptness of turns and trajectories very close to the obstacles. This makes it impossible to use the algorithm as a trajectory planner in real robotics. The main change in FM<sup>2</sup> [20] solves these problems by generating a speed map that modifies the expansion of the wave taking into account the

Topological navigation refers to the navigational processes that take place using a topological representation of the environment. A topological representation is characterized by defining reference elements of the environment according to the different relations between them. Reference elements are denominated nodes and the relations between them are characterized as arches. The aim of topological navigation is to develop navigational behaviors that are closer to those of humans in order to enhance the human-robot understanding. Summing up

• Topological navigation permits efficient planning. However, as they are based on rela-

• Huge diagrams are involved in large environments and diagrams scale better than geo-

tions, they do not minimize distance traveled or execution time.

• Topological navigation does not require precise localization.

• It is easy to understand by humans due to its natural conception.

iteratively (Figure 3).

Figure 3. Trajectory calculated by fast marching square.

proximity to obstacles.

3. Topological navigation

the main implications of topological navigation [21]:

• A complex recognition model is needed.

metrical representations.

#### 2.3. Fast marching square algorithm

Fast marching square (FM<sup>2</sup> ) is a path-planning algorithm, that searches for the optimal path between two points. It uses the fast marching method (FMM) as a basis for calculation. It is a modeling algorithm of a physical wave propagation.

The fast marching method uses a function that behaves similar to the propagation of a wave. The form that this wave propagates, following the Eikonal equation, from an initial point to reach a goal position is the most efficient way in terms of time to reach it. The fast marching algorithm calculates the time (T) that the front of the wave, called interface, spends to reach each point of the map from a starting point. The FMM requires as previous step a

Figure 2. PRM generated and trajectory created using bidirectional RRT. (a) PRM generated for a real environment and (b) Trajectory created using Bidirectional Rapidly Exploring Ramdom Trees.

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation http://dx.doi.org/10.5772/intechopen.79842 93

Figure 3. Trajectory calculated by fast marching square.

One of the main problems of PRM is that the solutions it finds do not have to be the optimal trajectory. Also, since the way it generates the nodes is completely random, it produces a nonhomogeneous distribution of the samples in the space. These two disadvantages are seen

Rapidly exploring random tree (RRT) [18] provides a solution by creating random branches from a starting point. The collision-free branches are stored iteratively and new ones are created until the target point is reached. The algorithm is started with a tree whose source is a single node, the starting point. In each iteration, the tree expands by selecting a random state and expanding the tree to that state. The expansion is performed by extending the nearest node of the tree to the selected random state, which will depend on the size of the selected step. The algorithm creates branches until the tree takes a certain extension approaching the goal.

The size of the step is an important parameter of the algorithm. Small values result in slow

between two points. It uses the fast marching method (FMM) as a basis for calculation. It is a

The fast marching method uses a function that behaves similar to the propagation of a wave. The form that this wave propagates, following the Eikonal equation, from an initial point to reach a goal position is the most efficient way in terms of time to reach it. The fast marching algorithm calculates the time (T) that the front of the wave, called interface, spends to reach each point of the map from a starting point. The FMM requires as previous step a

Figure 2. PRM generated and trajectory created using bidirectional RRT. (a) PRM generated for a real environment and

) is a path-planning algorithm, that searches for the optimal path

expansion, but with finer paths or paths that can take fine turns [19].

in Figure 2a.

92 Applications of Mobile Robots

2.2.2. Rapidly exploring random tree algorithm

2.3. Fast marching square algorithm

modeling algorithm of a physical wave propagation.

(b) Trajectory created using Bidirectional Rapidly Exploring Ramdom Trees.

Fast marching square (FM<sup>2</sup>

discretization of the environment in a grid of cells, on which the Eikonal equation is solved iteratively (Figure 3).

The trajectories obtained using FMM present two drawbacks: great abruptness of turns and trajectories very close to the obstacles. This makes it impossible to use the algorithm as a trajectory planner in real robotics. The main change in FM<sup>2</sup> [20] solves these problems by generating a speed map that modifies the expansion of the wave taking into account the proximity to obstacles.

#### 3. Topological navigation

Topological navigation refers to the navigational processes that take place using a topological representation of the environment. A topological representation is characterized by defining reference elements of the environment according to the different relations between them. Reference elements are denominated nodes and the relations between them are characterized as arches. The aim of topological navigation is to develop navigational behaviors that are closer to those of humans in order to enhance the human-robot understanding. Summing up the main implications of topological navigation [21]:


Topological representation classifications, as the one proposed by Vale & Ribeiro [22], differentiate mainly two ways of representing the environment: topological representations based on movements, for example the works developed by Kuipers & Byun [23], and topological representations based on geometrical maps, as proposed by Thrun [24]. These two conceptions differ mainly in the spatial relation between the real world and its representation. Regarding topological maps based on geometry, an exact relation between the environment and the representation is mandatory. Every topological node is metrically associated with a position or place in the environment, whereas, in topological representations based on movements, it is not necessary to have a metrical correspondence with the elements of the environment. An example of the same environment represented as a topological map based on geometry and as a topological map based on movements is shown in Figure 4.

representation based on geometry or a topological representation based on movements. In the case of using representations based on geometry, exploration and map acquisition strategies such as the ones explained previously for geometrical representations can be used adding a processing stage to extract the relevant geometrical positions. In the case of representations based on movements, strategies like Next Best View, as proposed in Amigoni & Gallo [26], or Frontier Exploration, as in the work proposed by Arvanitakis et al. [27] can be used. Generally, these representations are translated into a text file that lists the information of the graph in order to maximize the graph efficiency. In Figure 5, an example of a map text file structure and its graphical interpretation is shown. In this example, a topological representation based on movements is used so the position of the nodes in the graph is not necessarily linked to the

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

95

This text file contains all the information required for navigation. Nodes are ordered according to an identifier and they are associated with their corresponding event type (odometry, marker, closet, etc.). Arches are associated with the behavior or translational ability that the robot has

A model of the environment can be formed by several topological graphs containing different information or representing different levels of abstraction, in that case the environment is

Topological navigation behaviors are determined mainly by the path-planning and navigation (meaning strict motor abilities execution) strategies as considered in Mataric [28]. The different nodes conforming the path are associated with the topological place where the robot perceives a sensorial stimulus or where it reaches a required position. In addition, while navigating, the behavior of the robot is subjugated to real-time perception and movement in the environment. Topological planification is in charge of finding the topological route that the robot has to follow; it has to determine the path that the robot has to perform in order to move between two

position of reference elements in the environment.

to perform (GP, go to point; R, turn; etc.).

modeled as a hierarchical topological graph.

3.2. Topological planification and navigation

Figure 5. Example of a topological map and its graphical representation.

In topological representations based on geometry, nodes normally correspond to geometrical positions (characterized as ð Þ x; y; θ ) that correspond to geometrical events such as junctions, dead-ends, etc. and arches correspond to the geometrical transition between positions. In topological representations based on movements, the relation between nodes is more abstract as it does not have a geometrical meaning, being a qualitative relation instead of a quantitative one. That is why arches can be associated to different and more complex behaviors. In addition, nodes are associated to important sensorial events which can be determined for each application ranging from important geometrical events to landmarks and objects.

Although there are substantial differences between topological representations based on geometry and topological representations based on movements, both of them share the definition of the required modules so the system works efficiently. These modules will be explained in the following subsections, which are: modeling of the environment, planification and navigation, and perception interface.

#### 3.1. Modeling of the environment as a topological graph

A topological graph, as defined by Simhon & Dudek [25], is a graph representation of an environment in which the important elements of an environment are defined along with the transitions among them. The topological graph can be given to the system by the user or can be built through exploration. Exploration strategies differ if the system works with a topological

Figure 4. Topological representations based on geometry (left) and on movements (right).

representation based on geometry or a topological representation based on movements. In the case of using representations based on geometry, exploration and map acquisition strategies such as the ones explained previously for geometrical representations can be used adding a processing stage to extract the relevant geometrical positions. In the case of representations based on movements, strategies like Next Best View, as proposed in Amigoni & Gallo [26], or Frontier Exploration, as in the work proposed by Arvanitakis et al. [27] can be used. Generally, these representations are translated into a text file that lists the information of the graph in order to maximize the graph efficiency. In Figure 5, an example of a map text file structure and its graphical interpretation is shown. In this example, a topological representation based on movements is used so the position of the nodes in the graph is not necessarily linked to the position of reference elements in the environment.

This text file contains all the information required for navigation. Nodes are ordered according to an identifier and they are associated with their corresponding event type (odometry, marker, closet, etc.). Arches are associated with the behavior or translational ability that the robot has to perform (GP, go to point; R, turn; etc.).

A model of the environment can be formed by several topological graphs containing different information or representing different levels of abstraction, in that case the environment is modeled as a hierarchical topological graph.

#### 3.2. Topological planification and navigation

Topological representation classifications, as the one proposed by Vale & Ribeiro [22], differentiate mainly two ways of representing the environment: topological representations based on movements, for example the works developed by Kuipers & Byun [23], and topological representations based on geometrical maps, as proposed by Thrun [24]. These two conceptions differ mainly in the spatial relation between the real world and its representation. Regarding topological maps based on geometry, an exact relation between the environment and the representation is mandatory. Every topological node is metrically associated with a position or place in the environment, whereas, in topological representations based on movements, it is not necessary to have a metrical correspondence with the elements of the environment. An example of the same environment represented as a topological map based on geometry and as

In topological representations based on geometry, nodes normally correspond to geometrical positions (characterized as ð Þ x; y; θ ) that correspond to geometrical events such as junctions, dead-ends, etc. and arches correspond to the geometrical transition between positions. In topological representations based on movements, the relation between nodes is more abstract as it does not have a geometrical meaning, being a qualitative relation instead of a quantitative one. That is why arches can be associated to different and more complex behaviors. In addition, nodes are associated to important sensorial events which can be determined for each

Although there are substantial differences between topological representations based on geometry and topological representations based on movements, both of them share the definition of the required modules so the system works efficiently. These modules will be explained in the following subsections, which are: modeling of the environment, planification and navi-

A topological graph, as defined by Simhon & Dudek [25], is a graph representation of an environment in which the important elements of an environment are defined along with the transitions among them. The topological graph can be given to the system by the user or can be built through exploration. Exploration strategies differ if the system works with a topological

application ranging from important geometrical events to landmarks and objects.

a topological map based on movements is shown in Figure 4.

3.1. Modeling of the environment as a topological graph

Figure 4. Topological representations based on geometry (left) and on movements (right).

gation, and perception interface.

94 Applications of Mobile Robots

Topological navigation behaviors are determined mainly by the path-planning and navigation (meaning strict motor abilities execution) strategies as considered in Mataric [28]. The different nodes conforming the path are associated with the topological place where the robot perceives a sensorial stimulus or where it reaches a required position. In addition, while navigating, the behavior of the robot is subjugated to real-time perception and movement in the environment.

Topological planification is in charge of finding the topological route that the robot has to follow; it has to determine the path that the robot has to perform in order to move between two

Figure 5. Example of a topological map and its graphical representation.

nodes. The path is a subgraph of the original graph of the environment. In order to plan a trajectory in a topological representation of the environment, a graph path-planning algorithm has to be used. There are many algorithms in the literature for this purpose, such as Dijkstra's algorithm, Skiena [29]. The main objective of Dijkstra's algorithm is to obtain the shortest path between nodes in a graph according to some heuristics or cost function. Given an initial node, it evaluates adjacent nodes and chooses the node that minimizes the cost function. This process is iterated until the goal node is reached or every connection between nodes has been explored. Using traditional algorithms such as Dijkstra, many modifications can be implemented to establish heuristics that fit better to real environments. For example, the cost of a translation between nodes can be varied according to previous executions or pursing a specific global behavior, such as the personality factor proposed in Egido et al. [30].

simple. Perceptions are mainly based on vision, such as object detection and landmark detection, but some perceptions such as magnetic signals, ultrasounds, or proprioceptive perceptions such as odometry can be used. Another advantage of the perception interface is the

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

97

The current tendency in robotics is to move from representation models that are closest to the robot´s hardware such as geometric models to those models closer to the way how humans reason, with which the robot will interact. It is intended to bring closer the models the way robots represent the environment and the way they plan to the way the humans do. The current trend is to implement behavior mechanisms based on human psychology. Robots are provided with cognitive architectures in order to model the environment, using semantics concepts that provide more autonomy, and which helps the navigation to be robust and more

In theory, any characteristic of the space can be represented on a map. But in general, it tends to identify a map with geometric information more or less complemented with additional information. When a mobile robot builds a map, the techniques used generally ignore relevant descriptive information of the environment and quite close to the process of made by humans, such as the navigation of the environment, the nature of the activity that there is, what objects

The problem of the construction of semantic maps consists of maps that represent not only the occupation and geometric appearance of the environment but the same properties. Semantics is needed to give meaning to the data, mainly to make the simplest interpretation of data. The application of this idea to the construction of maps for mobile robots allows a better understanding of the data used to represent the environment and also allows an exchange of information between robots or between robots and people, if needed, easily. It also allows to

Semantic navigation is another step to the logical representation, it implies an additional knowledge about the elements of the world and it allows the robot to infer new information. For instance, the root can perform this type of associations: "There is a photocopier in this

Semantic navigation allows the robot to relate what it perceives to the places in which it is located. This way, an environment model is managed using the objects and on the concepts that they represent. All this information is used to classify the place where objects are located and it is used to reach a given location or a specific target. Therefore, the robot can find places or rooms Vasudevan & Siegwart [31] semantically related with the target, even if it is in a littleexplored or unknown environment. This navigation level allows to complete the information

Semantic navigation is related to classification methods of the environment object and places, which provides more effectiveness, as is described in Vasudevan & Siegwart [32]. An example

build more accurate models and more useful environment models.

that is necessary from a partial knowledge of the environment.

room-> there must be paper nearby".

possibility of establishing priorities and relations between perceptions easily.

4. Semantical navigation

efficient.

it contains, etc.

The navigator is in charge of performing the path and reaching the goal node analyzing events and abilities or positions and transitions. An ability is the order the robot has to execute to reach coming nodes and it is intrinsically related to motor control. An event is the sign indicating the robot has reached a node, through sensorial information or odometrically in the case of representations based on geometry. A navigator can be based on a single behavior or on multiple behaviors. Topological representations based on geometry are all based on a single behavior that can be understood as "Going to point" or "Reaching next position." Topological representations based on movements can be based on single behaviors also or define a set of behaviors that would be used for optimizing the transitions between nodes. A multiplebehavior system contains several strategies and abilities in order to perform navigation behaviors and it should enable the inclusion of new strategies. Some of the common abilities implemented are Go to point, Turn, Contour following, and Go to object. Navigation systems are complemented with reactive obstacle avoidance modules to guarantee safe operation.

#### 3.3. Managing the information of the environment: perception interface

If the topological navigation system is designed to work with multiple exteroceptive and proprioceptive events and it has to handle them simultaneously, these events have to be managed carefully. In order to manage the information that will be assigned to the nodes, a new module is needed. In this chapter, we will refer to this module as the perception interface.

The perception interface is decoupled from the system translating the specific information of each perception to a general structure which interacts with the other modules of the topological system, as shown in Figure 6. When translating the information to a general structure, the power of the system is multiplied exponentially, as adding new perception types becomes very

Figure 6. Perception interface and event manager modules distribution.

simple. Perceptions are mainly based on vision, such as object detection and landmark detection, but some perceptions such as magnetic signals, ultrasounds, or proprioceptive perceptions such as odometry can be used. Another advantage of the perception interface is the possibility of establishing priorities and relations between perceptions easily.

### 4. Semantical navigation

nodes. The path is a subgraph of the original graph of the environment. In order to plan a trajectory in a topological representation of the environment, a graph path-planning algorithm has to be used. There are many algorithms in the literature for this purpose, such as Dijkstra's algorithm, Skiena [29]. The main objective of Dijkstra's algorithm is to obtain the shortest path between nodes in a graph according to some heuristics or cost function. Given an initial node, it evaluates adjacent nodes and chooses the node that minimizes the cost function. This process is iterated until the goal node is reached or every connection between nodes has been explored. Using traditional algorithms such as Dijkstra, many modifications can be implemented to establish heuristics that fit better to real environments. For example, the cost of a translation between nodes can be varied according to previous executions or pursing a specific global

The navigator is in charge of performing the path and reaching the goal node analyzing events and abilities or positions and transitions. An ability is the order the robot has to execute to reach coming nodes and it is intrinsically related to motor control. An event is the sign indicating the robot has reached a node, through sensorial information or odometrically in the case of representations based on geometry. A navigator can be based on a single behavior or on multiple behaviors. Topological representations based on geometry are all based on a single behavior that can be understood as "Going to point" or "Reaching next position." Topological representations based on movements can be based on single behaviors also or define a set of behaviors that would be used for optimizing the transitions between nodes. A multiplebehavior system contains several strategies and abilities in order to perform navigation behaviors and it should enable the inclusion of new strategies. Some of the common abilities implemented are Go to point, Turn, Contour following, and Go to object. Navigation systems are complemented with reactive obstacle avoidance modules to guarantee safe operation.

If the topological navigation system is designed to work with multiple exteroceptive and proprioceptive events and it has to handle them simultaneously, these events have to be managed carefully. In order to manage the information that will be assigned to the nodes, a new module is needed. In this chapter, we will refer to this module as the perception interface. The perception interface is decoupled from the system translating the specific information of each perception to a general structure which interacts with the other modules of the topological system, as shown in Figure 6. When translating the information to a general structure, the power of the system is multiplied exponentially, as adding new perception types becomes very

behavior, such as the personality factor proposed in Egido et al. [30].

96 Applications of Mobile Robots

3.3. Managing the information of the environment: perception interface

Figure 6. Perception interface and event manager modules distribution.

The current tendency in robotics is to move from representation models that are closest to the robot´s hardware such as geometric models to those models closer to the way how humans reason, with which the robot will interact. It is intended to bring closer the models the way robots represent the environment and the way they plan to the way the humans do. The current trend is to implement behavior mechanisms based on human psychology. Robots are provided with cognitive architectures in order to model the environment, using semantics concepts that provide more autonomy, and which helps the navigation to be robust and more efficient.

In theory, any characteristic of the space can be represented on a map. But in general, it tends to identify a map with geometric information more or less complemented with additional information. When a mobile robot builds a map, the techniques used generally ignore relevant descriptive information of the environment and quite close to the process of made by humans, such as the navigation of the environment, the nature of the activity that there is, what objects it contains, etc.

The problem of the construction of semantic maps consists of maps that represent not only the occupation and geometric appearance of the environment but the same properties. Semantics is needed to give meaning to the data, mainly to make the simplest interpretation of data. The application of this idea to the construction of maps for mobile robots allows a better understanding of the data used to represent the environment and also allows an exchange of information between robots or between robots and people, if needed, easily. It also allows to build more accurate models and more useful environment models.

Semantic navigation is another step to the logical representation, it implies an additional knowledge about the elements of the world and it allows the robot to infer new information. For instance, the root can perform this type of associations: "There is a photocopier in this room-> there must be paper nearby".

Semantic navigation allows the robot to relate what it perceives to the places in which it is located. This way, an environment model is managed using the objects and on the concepts that they represent. All this information is used to classify the place where objects are located and it is used to reach a given location or a specific target. Therefore, the robot can find places or rooms Vasudevan & Siegwart [31] semantically related with the target, even if it is in a littleexplored or unknown environment. This navigation level allows to complete the information that is necessary from a partial knowledge of the environment.

Semantic navigation is related to classification methods of the environment object and places, which provides more effectiveness, as is described in Vasudevan & Siegwart [32]. An example of place identification using a Naive Bayes Classifier to infer place identification is shown in Duda et al. [33] and Kollar & Roy [34]. These works show that relations between object-object and objet-scenario can be used to predict the location of a variety of objects and scenarios.

• PhysicalRoom: It stores specific locations, places in space, regions delimited by walls,

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

99

• PhysicalObject: This table stores each real object that the robot sensor identifies. This

With these tables, objects and places have been modeled. More tables are needed in order to

Objects are always in rooms; so, between PhysicalRoom and PhysicalObject tables, there exists a link. And a physical room may have an indeterminate amount of physical objects. In table

As an example of the implementation of the tables, Figure 7b shows the design of the database. This database contains several tables that manage important information to help the robot to find objects and relations. With the information of this table, if the robot does not locate a specific object, it can deduce with what other objects it is linked to, helping the robot locate the

• Interaction: Objects interact with other objects. This table can handle any type of interaction, although tests have been performed with a limited number of them. The interactions taken into account are: BE\_A, when an object is a subtype of another object; IS\_USED\_WITH, when an object is used with an object; and IS\_INSIDE\_OF, when usually

• Utility: All objects have one or more utilities. The tendency is to group objects that have the same utility (or something related) in the same places. Also, concepts are created

• Meaning: The actions or utilities that an object has often are associated with a specific meaning. The goal of the navigation may also be oriented to emotions or places that give a

• Characteristic: Objects may have features to best define the concept or even differentiate them from others. For example, on the one hand, water may be cold, warm, or hot. And that implies differences in location. Cold water may be found in the refrigerator or in a

These tables include the semantic information of the objects in the environment that can help the robot to locate on it. For example, considering the utility, it can be said that a computer is

complete the relations and information of the environment that are described below.

To complete the model, the following links between the primary tables are needed:

object. To get additional information about objects, some tables are defined:

regarding kinds of room depending on what they are used for.

feeling, such as calm or fun. For example, read is an action quiet.

used to work. The robot can go to an office if it needs to locate a computer.

fountain, but hot water would come out from the faucet.

having real coordinates, etc., where the robot can move.

PhysicalObject, the room associated to each object is also stored.

4.2.3. Tables that allow searches by semantic proximity

an object is inside another object.

4.2.2. Links between primary tables

table contains specific objects: Table-1, Table-2, Computer-1, etc.

#### 4.1. Knowledge representation of the environment

One of the most important issues is how the robot models the environment. In semantic navigation, a large amount of information related to the environment and to the objects of the environment is used in the environment representation. For this representation, an ontology can be used to define the concepts and the links between objects in the environment. Figure 7a shows the concepts of the ontology and its relations.

#### 4.2. An ontology design through a database

The ontology design can be implemented using different tools. In this work, a relational model using a database is proposed. Figure 7b shows a relational database diagram scheme. This scheme is used to understand the elements of the system and its relations. Tables used to model the ontology are described below.

#### 4.2.1. Basic information tables

Basic information tables are used to store the environment elements. Four elements have been considered: physical rooms (real locations sensorially perceived by the robot), conceptual rooms (the type of room that the robot can recognize), physical objects (objects perceived by sensors), and conceptual objects (each semantic information of the object that the robot must recognize).


Figure 7. Ontological model for semantic navigation and its implementation on a database. (a) Ontological model for semantic navigation and (b) Database design for semantic navigation.


With these tables, objects and places have been modeled. More tables are needed in order to complete the relations and information of the environment that are described below.

#### 4.2.2. Links between primary tables

of place identification using a Naive Bayes Classifier to infer place identification is shown in Duda et al. [33] and Kollar & Roy [34]. These works show that relations between object-object and objet-scenario can be used to predict the location of a variety of objects and scenarios.

One of the most important issues is how the robot models the environment. In semantic navigation, a large amount of information related to the environment and to the objects of the environment is used in the environment representation. For this representation, an ontology can be used to define the concepts and the links between objects in the environment. Figure 7a

The ontology design can be implemented using different tools. In this work, a relational model using a database is proposed. Figure 7b shows a relational database diagram scheme. This scheme is used to understand the elements of the system and its relations. Tables used to

Basic information tables are used to store the environment elements. Four elements have been considered: physical rooms (real locations sensorially perceived by the robot), conceptual rooms (the type of room that the robot can recognize), physical objects (objects perceived by sensors), and conceptual objects (each semantic information of the object that the robot must recognize). • ConceptualRoom: This table stores the information of the rooms understood as concepts. The table records are those corresponding to the concepts bathroom, bed room, kitchen, etc.

• ConceptualObject: This table contains data related to objects abstractly defined and

Figure 7. Ontological model for semantic navigation and its implementation on a database. (a) Ontological model for

semantic navigation and (b) Database design for semantic navigation.

understood as concepts. For example, Oven, Computer, Washbasin, etc.

4.1. Knowledge representation of the environment

shows the concepts of the ontology and its relations.

4.2. An ontology design through a database

model the ontology are described below.

4.2.1. Basic information tables

98 Applications of Mobile Robots

To complete the model, the following links between the primary tables are needed:

Objects are always in rooms; so, between PhysicalRoom and PhysicalObject tables, there exists a link. And a physical room may have an indeterminate amount of physical objects. In table PhysicalObject, the room associated to each object is also stored.

#### 4.2.3. Tables that allow searches by semantic proximity

As an example of the implementation of the tables, Figure 7b shows the design of the database. This database contains several tables that manage important information to help the robot to find objects and relations. With the information of this table, if the robot does not locate a specific object, it can deduce with what other objects it is linked to, helping the robot locate the object. To get additional information about objects, some tables are defined:


These tables include the semantic information of the objects in the environment that can help the robot to locate on it. For example, considering the utility, it can be said that a computer is used to work. The robot can go to an office if it needs to locate a computer.

This relational database model provides a simple way to define and manage semantic knowledge using simple queries without the need to define rules as it has been described in previous works (Galindo et al. [35], Galindo et al. [36]).

#### 4.3. Semantic information management

Once the environment and its components are modeled, the next step is to manage the information in order to accomplish navigation tasks. Robots need to localize in the environment and calculate the path to the targets defined with semantical information. In the real robot, semantic targets must be transferred to the topological and geometrical levels in order to complete the movement of the robot.

#### 4.3.1. Semantic target obtention

Similar to the previous navigation system, a way to establish how to reach a target is needed. In the proposed semantic system is defined the concept of Semantic Position as a contextual information unit related to the robot's position. Semantic Position contains attributes that correspond to a room code (the target room in this case) and to an object code (the target object in this case). It is therefore understood that a position is obtained in relation to a semantic reference point, which can be either objects or rooms. The target of the robot is defined as a Semantic Position.

computer concept. The computer concept is associated with the watching movies utility and the watching movies utility is associated with the fun concept. Using the knowledge of Figure 8b, searches for objects with characteristics and actions can be done. If the destination is to drink something cold, the system recognizes cold water as a valid objective. The cold water is related to the refrigerator, the refrigerator is related to the kitchen, and the kitchen is related to Room-1.

Figure 8. Ontology concepts and semantic relations. (a) Ontology concepts and semantic relations of a office and (b)

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

101

A place, in this case a room, can be identified as a vector of detected objects. The object observed from the environment must be analyzed and with a query we can get a list of the types of room where the object can be found. If an object is found, it defines the room where it is located. For instance, the bed is an object that can only be in a bedroom. If the result of the previous query returns several records, it is because the object is not discriminatory enough to be able to conclude any identification, and more additional semantic information and more

In this example, the robot is asked for several targets in a home. In the experiment, the robot has the information provided by the previous exploration of the environment. This information is stored in several tables: 1a is the table with physical objects information, 1a is the table

The objective is to use semantic information to get the semantic target, that is attached to a

with physical objects information and 1b is the table with deducted knowledge.

topological or to a geometrical target to which the robot must move (Table 1).

Then, the system goes to Room-1.

Ontology concepts and semantic relations of a kitchen.

4.3.2. Semantic identification of a place

4.4. Example

queries are needed to get the destination room.

Depending on the information available in the database about the requested destination and the type of destination (object or room), several options can be found in order to establish the semantic path:


To get the Semantic Target, several consultations to the database must be done. One query is used to obtain the specific object instances that have already been stored, other query to know the physical room where a specific object is found, and a last query to match a room with a specific type of object.

In addition, this system allows the search of destinations by semantic proximity. For example, using the knowledge of Figure 8a if the requested objective is to do something fun, the answer would be to go to the computer-1. This is because the computer-1 is an object associated with the Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation http://dx.doi.org/10.5772/intechopen.79842 101

Figure 8. Ontology concepts and semantic relations. (a) Ontology concepts and semantic relations of a office and (b) Ontology concepts and semantic relations of a kitchen.

computer concept. The computer concept is associated with the watching movies utility and the watching movies utility is associated with the fun concept. Using the knowledge of Figure 8b, searches for objects with characteristics and actions can be done. If the destination is to drink something cold, the system recognizes cold water as a valid objective. The cold water is related to the refrigerator, the refrigerator is related to the kitchen, and the kitchen is related to Room-1. Then, the system goes to Room-1.

#### 4.3.2. Semantic identification of a place

A place, in this case a room, can be identified as a vector of detected objects. The object observed from the environment must be analyzed and with a query we can get a list of the types of room where the object can be found. If an object is found, it defines the room where it is located. For instance, the bed is an object that can only be in a bedroom. If the result of the previous query returns several records, it is because the object is not discriminatory enough to be able to conclude any identification, and more additional semantic information and more queries are needed to get the destination room.

#### 4.4. Example

This relational database model provides a simple way to define and manage semantic knowledge using simple queries without the need to define rules as it has been described in previous

Once the environment and its components are modeled, the next step is to manage the information in order to accomplish navigation tasks. Robots need to localize in the environment and calculate the path to the targets defined with semantical information. In the real robot, semantic targets must be transferred to the topological and geometrical levels in order to

Similar to the previous navigation system, a way to establish how to reach a target is needed. In the proposed semantic system is defined the concept of Semantic Position as a contextual information unit related to the robot's position. Semantic Position contains attributes that correspond to a room code (the target room in this case) and to an object code (the target object in this case). It is therefore understood that a position is obtained in relation to a semantic reference point, which can be either objects or rooms. The target of the robot is defined as a

Depending on the information available in the database about the requested destination and the type of destination (object or room), several options can be found in order to establish the

• If the destination is a known object in a known room, the robot has all the information to

• If the destination is an unknown room, a semantic exploration routine must be used to manage the semantic information to locate a room with similar semantic information of

• If the destination is an unknown object in a known room, the robot would try to explore it to get an object with similar semantic information in the room to define it as a Semantic Target.

• If the destination is an unknown object in an unknown room, all information of the object and room in the database must be used to get an object in a room which matches with the

To get the Semantic Target, several consultations to the database must be done. One query is used to obtain the specific object instances that have already been stored, other query to know the physical room where a specific object is found, and a last query to match a room with a

In addition, this system allows the search of destinations by semantic proximity. For example, using the knowledge of Figure 8a if the requested objective is to do something fun, the answer would be to go to the computer-1. This is because the computer-1 is an object associated with the

works (Galindo et al. [35], Galindo et al. [36]).

4.3. Semantic information management

complete the movement of the robot.

4.3.1. Semantic target obtention

100 Applications of Mobile Robots

reach the Semantic Target.

semantical information of the destination.

the destination.

specific type of object.

Semantic Position.

semantic path:

In this example, the robot is asked for several targets in a home. In the experiment, the robot has the information provided by the previous exploration of the environment. This information is stored in several tables: 1a is the table with physical objects information, 1a is the table with physical objects information and 1b is the table with deducted knowledge.

The objective is to use semantic information to get the semantic target, that is attached to a topological or to a geometrical target to which the robot must move (Table 1).

First, the robot is asked for the kitchen. Checking the table PhysicalConceptualRoom with a query, the result is Room-1. Repeating the query, in the tables, there are no more kitchens in the database, so the process finishes.

In a second experiment, the robot is asked for a chair. In the initial environment, there are three real objects of the CHAIR type. In the query, the robot identifies three chairs: Chair-1, Chair-2, and Chair-3. The robot gets the information of the rooms in which the chairs are and moves to the first option. If it indicates that this is not the correct room, the robot moves to the rest of the options. This process is described in Figure 9.

In case the object has no match with the observed physical object, it is necessary to ask again for a chair, after having discarded the chairs from the previous test. The operation sequence is shown in Figure 10a. In Figure 9, another query on chair is shown, and the robot starts to explore searching in new types of rooms where chairs could be found.


Table 1. Tables with semantic information.

5. Perception model for navigation

with no observed physical object and (b) Content of the table ConceptualObjectRoom.

Talking about autonomous robots implies carrying out movements safely and having a complete knowledge of the environment. These elements define the capabilities of action and interaction between the environment, humans, and robots. Tasks performed by mobile robots such as navigation, localization, planning, among others, can be improved if the perceptual information is considered. So, the general idea is detecting and identifying meaningful elements (objects and scenes) of the environment. There are different ways to obtain the information about the environment. One of them consists of detecting recognizable features of the environment (natural landmarks) using several types of sensors. The detection of artificial landmarks, based on the work of Fischler & Elschlager [37], can be used to acquire a representation of the environment. However, due to the technological advances in 3D sensors (e.g., RGB-D sensors) and according to Huang [38], vision has become the main sensor used for navigation, localization, and scene recognition. Vision provides significant information about

Figure 10. Ontology concepts and semantic relations. (a) Test sequence with the CHAIR object when there is no match

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

103

Figure 9. Test sequence with the CHAIR object.

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation http://dx.doi.org/10.5772/intechopen.79842 103

Figure 10. Ontology concepts and semantic relations. (a) Test sequence with the CHAIR object when there is no match with no observed physical object and (b) Content of the table ConceptualObjectRoom.

#### 5. Perception model for navigation

First, the robot is asked for the kitchen. Checking the table PhysicalConceptualRoom with a query, the result is Room-1. Repeating the query, in the tables, there are no more kitchens in

In a second experiment, the robot is asked for a chair. In the initial environment, there are three real objects of the CHAIR type. In the query, the robot identifies three chairs: Chair-1, Chair-2, and Chair-3. The robot gets the information of the rooms in which the chairs are and moves to the first option. If it indicates that this is not the correct room, the robot moves to the rest of the

In case the object has no match with the observed physical object, it is necessary to ask again for a chair, after having discarded the chairs from the previous test. The operation sequence is shown in Figure 10a. In Figure 9, another query on chair is shown, and the robot starts to

(a) Content of the table physical object (b) Content of the table conceptual physical room

Name Conceptual name Room name Conceptual name Physical name Chair-1 CHAIR Room-1 Room-1 KITCHEN Chair-2 CHAIR Room-2 Room-2 LIVING ROOM Chair-3 CHAIR Room-3 Room-3 OFFICE

explore searching in new types of rooms where chairs could be found.

Physical object Conceptual physical room

the database, so the process finishes.

102 Applications of Mobile Robots

options. This process is described in Figure 9.

Refrigerator-1 REFRIGERATOR Room-1 Sofa-1 SOFA Room-2 Desk-1 DESK Room-3

Table 1. Tables with semantic information.

Figure 9. Test sequence with the CHAIR object.

Talking about autonomous robots implies carrying out movements safely and having a complete knowledge of the environment. These elements define the capabilities of action and interaction between the environment, humans, and robots. Tasks performed by mobile robots such as navigation, localization, planning, among others, can be improved if the perceptual information is considered. So, the general idea is detecting and identifying meaningful elements (objects and scenes) of the environment. There are different ways to obtain the information about the environment. One of them consists of detecting recognizable features of the environment (natural landmarks) using several types of sensors. The detection of artificial landmarks, based on the work of Fischler & Elschlager [37], can be used to acquire a representation of the environment. However, due to the technological advances in 3D sensors (e.g., RGB-D sensors) and according to Huang [38], vision has become the main sensor used for navigation, localization, and scene recognition. Vision provides significant information about the objects present in a place; at the same time, it is capable of providing semantic information of the scene where it is. Scene recognition is a very well-known challenging issue that deals with how robots understand scenes just like a human does and the appearance variability of real environments. Therefore, regardless of the type of navigation used, whether geometrical, topological, or semantic, place recognition and object identification play a significant role in terms of representation of the environment.

more realistic way. The uncertainty calculation can be determined considering the dependency relations between different factors. Some of the factors are: the accuracy of the model that can be determined empirically and a factor based on the outcomes of the classification algorithm. Also, other factors can include the influence of other elements of the environment, for example the distance during the detection process. Finally, considering these factors makes it possible to

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

105

The growth of service robotics in recent years has created the needed for developing models that contribute to robots being able to adequately handle information from human environments. A semantic model can improve the high-level tasks of a robot, such as, semantic navigation and human-robot and robot-environment interaction. According to Wang et al. [44], semantic navigation is considered as a system that takes into account semantic information to represent the environment and then to carry out the localization and navigation of the

In view of the above, place recognition deals with the process of recognizing an area of the environment in which there are elements (objects), actions are developed, and robotenvironment and human-robot interaction is possible. The scene-understanding issue can be defined as a combination between scene recognition and object detection. There are approaches that try to solve the scene recognition problem through computer vision algorithms, including the creation of complex feature descriptors (Xie et al. [45]), and a combination of feature extraction techniques (Nicosevici & Garcia [46] and Khan et al. [47]), among others. Moreover, if it is desired to get a robust model as close as possible to reality, the

Some approaches are based on machine learning and select support vector machine as classification technique. Commonly, this type of place recognition model is composed by the following processes (Figure 11): image preprocessing, that includes the selection of the datasets and the initial preprocessing of the images to obtain the training model; feature extraction, that can implement some techniques such as bag of words, local and global descriptors, among others; training process, where the parameters of the classifier are defined and the model of the scene is generated; prediction process that generates the final results of the possible scene where the robot is located; and ,finally, a last process called reclassification has to be considered in which it is possible to generate a relationship between the place recognition model and

In this way, the influence of the objects in the scene can improve, worsen, and correct the final results on where the robot is located. The process implies the adjustment of the probabilities of being in a place and therefore the management of uncertainties. To do that, it is necessary that the place recognition model and the object recognition system work simultaneously in real time. The uncertainty can be modeled from a set of rules based on learning to determine the probability of co-occurrence of the objects. Also, it is important to incorporate the prior information of the place and object recognition models. Finally, it is possible to apply different

obtain a robust object recognition model to serve as an input to a mobile robot.

incorporation of environmental data and errors of the sensors is needed.

the elements (objects) of the environment.

5.2. Place recognition

robot.

#### 5.1. Object recognition

To perform several tasks in common indoor environments, mobile robots need to quickly and accurately verify and recognize objects, obstacles, etc. One of these crucial tasks is to move safely in an unknown environment. Autonomous robots should be able to acquire and hold visual representations of their environments.

The most important stages in an object recognition model are: feature extraction and prediction. As for feature extraction, the identification of significant aspects of different objects that belong to the same class, independently of the appearance variabilities, such a scaling, rotation, translation, illumination changing, among others, is crucial to obtain a suitable representation of the objects present in a scene. Some techniques are based on local and global descriptors such as the works presented by Bay et al. [39] and Hernández et al. [40], or a combination of both of them (Hernandez-Lopez et al. [41]). Also, in other approaches such as the work presented by Csurka et al. [42], visual vocabularies (e. g., bag of words) are commonly used to create a proper representation of each object.

Regarding prediction, through the vectors created from the extracted features, it is possible to learn these characteristics in order to identify objects that correspond with each class. In the literature, different classification techniques based on machine learning such as nearest neighbor classifier, neural networks, AdaBoost, etc., have been proposed depending on the kind of extracted features (Pontil & Verri [43]). Machine learning is a field of computer science that includes algorithms that improve their performance at a given task considering the experience. In this way, support vector machine (SVM) is one of the most helpful classification algorithms. The aim of SVM is to generate a training model that is capable of predicting the target classes of the test dataset, considering only its attributes.

Generally, an object recognition system that works in real time is divided into two stages: offline and online. Offline stage includes all the processes to reduce the execution time and guarantee the efficiency of the system, which are image preprocessing, segmentation, feature extraction, and training process. Online stage refers to the processes carried out in real time with the goal to ensure the interaction between the robot and the environment. Mainly, the processes included in the online stage are image retrieval and classification.

Object detection can be very useful for a navigation system since it allows the robot to relate what it perceives to the scenes in which it is. For this reason, it is necessary to consider that the designed systems and the sensors are not perfect. Therefore, the object detection model has to incorporate uncertainty information. The uncertainty management is a relevant aspect in an object recognition system because it allows to represent the environment and its elements in a more realistic way. The uncertainty calculation can be determined considering the dependency relations between different factors. Some of the factors are: the accuracy of the model that can be determined empirically and a factor based on the outcomes of the classification algorithm. Also, other factors can include the influence of other elements of the environment, for example the distance during the detection process. Finally, considering these factors makes it possible to obtain a robust object recognition model to serve as an input to a mobile robot.

#### 5.2. Place recognition

the objects present in a place; at the same time, it is capable of providing semantic information of the scene where it is. Scene recognition is a very well-known challenging issue that deals with how robots understand scenes just like a human does and the appearance variability of real environments. Therefore, regardless of the type of navigation used, whether geometrical, topological, or semantic, place recognition and object identification play a significant role in

To perform several tasks in common indoor environments, mobile robots need to quickly and accurately verify and recognize objects, obstacles, etc. One of these crucial tasks is to move safely in an unknown environment. Autonomous robots should be able to acquire and hold

The most important stages in an object recognition model are: feature extraction and prediction. As for feature extraction, the identification of significant aspects of different objects that belong to the same class, independently of the appearance variabilities, such a scaling, rotation, translation, illumination changing, among others, is crucial to obtain a suitable representation of the objects present in a scene. Some techniques are based on local and global descriptors such as the works presented by Bay et al. [39] and Hernández et al. [40], or a combination of both of them (Hernandez-Lopez et al. [41]). Also, in other approaches such as the work presented by Csurka et al. [42], visual vocabularies (e. g., bag of words) are com-

Regarding prediction, through the vectors created from the extracted features, it is possible to learn these characteristics in order to identify objects that correspond with each class. In the literature, different classification techniques based on machine learning such as nearest neighbor classifier, neural networks, AdaBoost, etc., have been proposed depending on the kind of extracted features (Pontil & Verri [43]). Machine learning is a field of computer science that includes algorithms that improve their performance at a given task considering the experience. In this way, support vector machine (SVM) is one of the most helpful classification algorithms. The aim of SVM is to generate a training model that is capable of predicting the target classes

Generally, an object recognition system that works in real time is divided into two stages: offline and online. Offline stage includes all the processes to reduce the execution time and guarantee the efficiency of the system, which are image preprocessing, segmentation, feature extraction, and training process. Online stage refers to the processes carried out in real time with the goal to ensure the interaction between the robot and the environment. Mainly, the

Object detection can be very useful for a navigation system since it allows the robot to relate what it perceives to the scenes in which it is. For this reason, it is necessary to consider that the designed systems and the sensors are not perfect. Therefore, the object detection model has to incorporate uncertainty information. The uncertainty management is a relevant aspect in an object recognition system because it allows to represent the environment and its elements in a

processes included in the online stage are image retrieval and classification.

terms of representation of the environment.

visual representations of their environments.

monly used to create a proper representation of each object.

of the test dataset, considering only its attributes.

5.1. Object recognition

104 Applications of Mobile Robots

The growth of service robotics in recent years has created the needed for developing models that contribute to robots being able to adequately handle information from human environments. A semantic model can improve the high-level tasks of a robot, such as, semantic navigation and human-robot and robot-environment interaction. According to Wang et al. [44], semantic navigation is considered as a system that takes into account semantic information to represent the environment and then to carry out the localization and navigation of the robot.

In view of the above, place recognition deals with the process of recognizing an area of the environment in which there are elements (objects), actions are developed, and robotenvironment and human-robot interaction is possible. The scene-understanding issue can be defined as a combination between scene recognition and object detection. There are approaches that try to solve the scene recognition problem through computer vision algorithms, including the creation of complex feature descriptors (Xie et al. [45]), and a combination of feature extraction techniques (Nicosevici & Garcia [46] and Khan et al. [47]), among others. Moreover, if it is desired to get a robust model as close as possible to reality, the incorporation of environmental data and errors of the sensors is needed.

Some approaches are based on machine learning and select support vector machine as classification technique. Commonly, this type of place recognition model is composed by the following processes (Figure 11): image preprocessing, that includes the selection of the datasets and the initial preprocessing of the images to obtain the training model; feature extraction, that can implement some techniques such as bag of words, local and global descriptors, among others; training process, where the parameters of the classifier are defined and the model of the scene is generated; prediction process that generates the final results of the possible scene where the robot is located; and ,finally, a last process called reclassification has to be considered in which it is possible to generate a relationship between the place recognition model and the elements (objects) of the environment.

In this way, the influence of the objects in the scene can improve, worsen, and correct the final results on where the robot is located. The process implies the adjustment of the probabilities of being in a place and therefore the management of uncertainties. To do that, it is necessary that the place recognition model and the object recognition system work simultaneously in real time. The uncertainty can be modeled from a set of rules based on learning to determine the probability of co-occurrence of the objects. Also, it is important to incorporate the prior information of the place and object recognition models. Finally, it is possible to apply different

Author details

References

2608

9(2):25

\*Address all correspondence to: rbarber@ing.uc3m.es

Robotics and Automation Magazine. 2006;13(3):108-117

University Carlos III of Madrid, Spain

Intelligence. 1990;44(3):305-360

Engineering. 2004;11(4):309-322

2016. pp. 1530-1537

Springer; 1990. pp. 420-443

Berlin Heidelberg: Springer; 2000. pp. 1-17

Ramón Barber\*, Jonathan Crespo, Clara Gómez, Alejandra C. Hernámdez and Marina Galli

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

107

[1] Levitt TS. Qualitative navigation for mobile robots. International Journal of Artificial

[2] Bailey T, Durrant-Whyte H. Simultaneous localization and mapping (slam): Part II. IEEE

[3] Wang C, Meng L, She S, Mitchell IM, Li T, Tung F, et al. Autonomous mobile robot navigation in uneven and unstructured indoor environments. In: Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on. IEEE; 2017. pp. 109-116

[4] Pfrunder A, Borges PV, Romero AR, Catt G, Elfes A. Real-time autonomous ground vehicle navigation in heterogeneous environments using a 3d lidar. In: Intelligent Robots and Systems (IROS), 2017 IEEE/RSJ International Conference on. IEEE; 2017. pp. 2601-

[5] Fernández-Madrigal J-A, Galindo C, González J. Assistive navigation of a robotic wheelchair using a multihierarchical model of the environment. Integrated Computer-Aided

[6] Ko DW, Kim YN, Lee JH, Suh IH. A scene-based dependable indoor navigation system. In: Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on. IEEE;

[7] Kuipers BJ, Levitt TS. Navigation and mapping in large scale space. AI Magazine. 1988;

[8] Giralt G, Chatila R, Vaisset M. An Integrated Navigation and Motion Control System for Autonomous Multisensory Mobile Robots, Autonomous Robot Vehicles. New York:

[9] Mataric MJ. A distributed model for mobile robot environment-learning and navigation.

[11] De Berg M, Van Kreveld M, Overmars M, Schwarzkopf OC. Computational Geometry.

Massachusetts Institute of Technology Cambridge. MA, USA; 1990

[10] LaValle SM. Planning Algorithms. Cambridge University Press; 2006

Figure 11. Schematic of a general scene recognition model. Vision data are the initial input of the recognition model. Then, the metric data and object information can be included into the model to update the final outcome of the possible place where the robot can be.

theorems such as Bayes to determine the final relation between objects and scenes, and the last result about where the robot is. The adjusted prediction has to be available as input for relevant tasks such as localization, navigation, scene understanding, and human-robot and robot-environment interaction. All this contributes to the main goal that is to have an autonomous and independent robot, with a wide knowledge of the environment.

#### 6. Conclusion

The aim of this chapter was to describe different approaches for global navigation systems for mobile robots applied to indoor environments. Many researches and current developments are focused on solving specific needs for navigation. Our objective is to merge all these developments in order to classify them and establish a global frame for navigation.

Robot navigation has been tackled from different perspectives leading to a classification into three main approaches: geometric navigation, topological navigation, and semantic navigation. Although the three of them differ in their definition and methods, all of them have the focus on driving a robot autonomously and safely.

In this chapter, different trends and techniques have been presented, all of them inspired by biological models and pursuing human abilities and abstraction models. The geometric representation, closer to the sensor and actuator world, is the best one to perform local navigation and precise path-planning. Topological representation of the environment, which is based on graphs, enables large navigation tasks and uses similar models as humans do. The semantic representation, which is the closest to cognitive human models, adds concepts such as utilities or meanings of the environment elements and establishes complex relations between them. All of these representations are based on the available information of the environment. For this reason, perception plays a significant role in terms of understanding and moving through the environment.

Despite the differences between the environment representations, we consider that the integration of all of them and the proper management of the information is the key to achieve a global navigation system.

#### Author details

Ramón Barber\*, Jonathan Crespo, Clara Gómez, Alejandra C. Hernámdez and Marina Galli

\*Address all correspondence to: rbarber@ing.uc3m.es

University Carlos III of Madrid, Spain

#### References

theorems such as Bayes to determine the final relation between objects and scenes, and the last result about where the robot is. The adjusted prediction has to be available as input for relevant tasks such as localization, navigation, scene understanding, and human-robot and robot-environment interaction. All this contributes to the main goal that is to have an autono-

Figure 11. Schematic of a general scene recognition model. Vision data are the initial input of the recognition model. Then, the metric data and object information can be included into the model to update the final outcome of the possible

The aim of this chapter was to describe different approaches for global navigation systems for mobile robots applied to indoor environments. Many researches and current developments are focused on solving specific needs for navigation. Our objective is to merge all these develop-

Robot navigation has been tackled from different perspectives leading to a classification into three main approaches: geometric navigation, topological navigation, and semantic navigation. Although the three of them differ in their definition and methods, all of them have the

In this chapter, different trends and techniques have been presented, all of them inspired by biological models and pursuing human abilities and abstraction models. The geometric representation, closer to the sensor and actuator world, is the best one to perform local navigation and precise path-planning. Topological representation of the environment, which is based on graphs, enables large navigation tasks and uses similar models as humans do. The semantic representation, which is the closest to cognitive human models, adds concepts such as utilities or meanings of the environment elements and establishes complex relations between them. All of these representations are based on the available information of the environment. For this reason, perception

plays a significant role in terms of understanding and moving through the environment.

Despite the differences between the environment representations, we consider that the integration of all of them and the proper management of the information is the key to achieve a

mous and independent robot, with a wide knowledge of the environment.

ments in order to classify them and establish a global frame for navigation.

focus on driving a robot autonomously and safely.

6. Conclusion

place where the robot can be.

106 Applications of Mobile Robots

global navigation system.


[12] Hwang YK, Ahuja N. A potential field approach to path planning. IEEE Transactions on Robotics and Automation. 1992;8(1):23-32

[28] Mataric MJ. Behaviour-based control: Examples from navigation, learning, and group behaviour. Journal of Experimental & Theoretical Artificial Intelligence. 1997;11:323-336

Mobile Robot Navigation in Indoor Environments: Geometric, Topological, and Semantic Navigation

http://dx.doi.org/10.5772/intechopen.79842

109

[29] Skiena S. Dijkstra's Algorithm. Implementing Discrete Mathematics: Combinatorics and Graph Theory with Mathematica. MA: Addison-Wesley, Reading; 1990. pp. 225-227 [30] Egido V, Barber R, Boada MJL, Salichs MA. A planner for topological navigation based on previous experiences. IFAC Proceedings Volumes; 2004;37(8):615-620. ISSN 1474-6670.

[31] Vasudevan S, Siegwart R. Bayesian space conceptualization and place classification for semantic maps in mobile robotics. Robotics and Autonomous Systems (RAS). 2008;56:

[32] Vasudevan S, Siegwart R. A bayesian conceptualization of space for mobile robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2007

[33] Duda RO, Hart PE, Stork DG. Pattern Classification. New York, NY, USA: Wiley Inters-

[34] Kollar T, Roy N. Utilizing object-object and object-scene context when planning to find things. In: IEEE International Conference on Robotics and Automation (ICRA). 2009 [35] Galindo C, Fernndez-Madrigal J-A, Gonzlez J, Saffiotti A. Robot task planning using

[36] Galindo C, Saffiotti A, Coradeschi S, Buschka P, Fernndez-Madrigal JA, Gonzlez J. Multi-hierarchical semantic maps for mobile robotics. In: Proceedings of IROS. 2005 [37] Fischler MA, Elschlager RA. The representation and matching of pictorial structures. IEEE

[38] Huang T. Computer vision: Evolution and promise. In: International conference; 5th, High

[39] Bay H, Tuytelaars T, Van Gool L. Surf: Speeded up robust features. In: European Confer-

[40] Hernández AC, Gómez C, Crespo J, Barber R. Object detection applied to indoor environ-

[41] Hernandez-Lopez J-J, Quintanilla-Olvera A-L, López-Ramírez J-L, Rangel-Butanda F-J, Ibarra-Manzano M-A, Almanza-Ojeda D-L. Detecting objects using color and depth seg-

[42] Csurka G, Dance C, Fan L, Willamowski J, Bray C. Visual categorization with bags of keypoints. In: Workshop on statistical learning in computer vision, ECCV, Vol. 1; Prague.

[43] Pontil M, Verri A. Support vector machines for 3d object recognition. IEEE Transactions on

semantic maps, Robotics and Autonomous Systems. 2008;56(11):955-966

technology: Imaging science and technology. Chiba, Japan; 1996. pp.13-20

https://doi.org/10.1016/S1474-6670(17)32046-3

Transactions on Computers. 1973;100(1):67-92

ence on Computer Vision. Springer; 2006. pp. 404-417

ments for mobile robot navigation. Sensors. 2016;16(8):1180

mentation with kinect sensor. Procedia Technology. 2012;3:196-204

Pattern Analysis and Machine Intelligence. 1998;20(6):637-646

522-537

cience; 2000

2004. pp. 1-2


[28] Mataric MJ. Behaviour-based control: Examples from navigation, learning, and group behaviour. Journal of Experimental & Theoretical Artificial Intelligence. 1997;11:323-336

[12] Hwang YK, Ahuja N. A potential field approach to path planning. IEEE Transactions on

[13] Poty A, Melchior P, Oustaloup A. Dynamic path planning for mobile robots using fractional potential field. In: Control, Communications and Signal Processing, 2004. First

[14] Lee D-T, Drysdale RL III. Generalization of voronoi diagrams in the plane. SIAM Journal

[15] Breu H, Gil J, Kirkpatrick D, Werman M. Linear time euclidean distance transform algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence. 1995;17(5):529-533

[16] Alt H, Cheong O, Vigneron A. The voronoi diagram of curved objects. Discrete & Com-

[17] Gang L, Wang J. Prm path planning optimization algorithm research. (n.d.). Wseas Trans-

[18] Kuffner JJ, LaValle SM. Rrt-connect: An efficient approach to single-query path planning. In: Robotics and Automation, 2000. Proceedings of ICRA'00. IEEE International Confer-

[20] Valero-Gomez A, Gomez JV, Garrido S, Moreno L. The path to efficiency: Fast marching method for safer, more efficient mobile robot trajectories. IEEE Robotics and Automation

[21] Thrun S, Bücken A. Integrating grid-based and topological maps for mobile robot navigation. In: Proceedings of the National Conference on Artificial Intelligence. 1996. pp. 944-951

[22] Vale A, Ribeiro M. Environment mapping as a topological representation. In: Proceedings of the 11th International Conference on Advanced Robotics-ICAR. 2003. pp. 1-3

[23] Kuipers BJ, Byun YT. A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations. Robotics and Autonomous Systems. 1991;8(1–2):47-63

[24] Thrun S. Learning metric-topological maps for indoor mobile robot navigation. Artificial

[25] Simhon S, Dudek G. A global topological map formed by local metric maps. In: IEEE/RSJ

[26] Amigoni F, Gallo A. A multi-objective exploration strategy for mobile robots. In: Robotics and Automation, 2005. ICRA 2005. Proceedings of the 2005 IEEE International Conference

[27] Arvanitakis I, Giannousakis K, Tzes A. Mobile robot navigation in unknown environment based on exploration principles. In: Control Applications (CCA), 2016 IEEE Conference

International Conference on Intelligent Robotic Systems. 1998. pp. 1708-1714

[19] LaValle SM. Rapidly-exploring random trees: A new tool for path planning. 1998

Robotics and Automation. 1992;8(1):23-32

on Computing. 1981;10(1):73-87

108 Applications of Mobile Robots

putational Geometry. 2005;34(3):439-453

ence on, Vol. 2. IEEE; 2000. pp. 995-1001

Magazine. 2013;20(4):111-120

Intelligence. 1998;99(1):21-71

on. IEEE; 2005. pp. 3850-3855

on. IEEE; 2016. pp. 493-498

International Symposium on. IEEE; 2004. pp. 557-561

actions on Systems and Control. 2016;11. E-ISSN: 2224-2856


[44] Wang L, Zhao L, Huo G, Li R, Hou Z, Luo P, et al. Visual semantic navigation based on deep learning for indoor mobile robots. Complexity. 2018;2018. Article ID: 1627185

**Chapter 6**

**Provisional chapter**

**Intelligent Robotic Perception Systems**

**Intelligent Robotic Perception Systems**

DOI: 10.5772/intechopen.79742

Robotic perception is related to many applications in robotics where sensory data and artificial intelligence/machine learning (AI/ML) techniques are involved. Examples of such applications are object detection, environment representation, scene understanding, human/pedestrian detection, activity recognition, semantic place classification, object modeling, among others. Robotic perception, in the scope of this chapter, encompasses the ML algorithms and techniques that empower robots to learn from sensory data and, based on learned models, to react and take decisions accordingly. The recent developments in machine learning, namely deep-learning approaches, are evident and, consequently, robotic perception systems are evolving in a way that new applications and tasks are becoming a reality. Recent advances in human-robot interaction, complex robotic tasks, intelligent reasoning, and decision-making are, at some extent, the results of the notorious evolution and success of ML algorithms. This chapter will cover recent and emerging topics and use-cases related to intelligent perception systems in robotics.

In robotics, perception is understood as a system that endows the robot with the ability to perceive, comprehend, and reason about the surrounding environment. The key components of a perception system are essentially sensory data processing, data representation (environment modeling), and ML-based algorithms, as illustrated in **Figure 1**. Since *strong AI* is still far from being achieved in real-world robotics applications, this chapter is about *weak AI*, i.e.,

**Keywords:** robotic perception, machine learning, advanced robotics,

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

Cristiano Premebida, Rares Ambrus and

Cristiano Premebida, Rares Ambrus and

http://dx.doi.org/10.5772/intechopen.79742

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

Zoltan-Csaba Marton

Zoltan-Csaba Marton

**Abstract**

artificial intelligence

standard machine learning approaches [1].

**1. Introduction**


#### **Intelligent Robotic Perception Systems Intelligent Robotic Perception Systems**

Cristiano Premebida, Rares Ambrus and Zoltan-Csaba Marton Cristiano Premebida, Rares Ambrus and Zoltan-Csaba Marton

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79742

#### **Abstract**

[44] Wang L, Zhao L, Huo G, Li R, Hou Z, Luo P, et al. Visual semantic navigation based on deep learning for indoor mobile robots. Complexity. 2018;2018. Article ID: 1627185 [45] Xie L, Tian Q, Wang M, Zhang B. Spatial pooling of heterogeneous features for image

[46] Nicosevici T, Garcia R. Automatic visual bag-of-words for online robot navigation and

[47] Khan SH, Bennamoun M, Sohel F, Togneri R. Geometry driven semantic labeling of indoor scenes. In: European Conference on Computer Vision. Springer; 2014. pp. 679-694

classification. IEEE Transactions on Image Processing. 2014;23(5):1994-2008

mapping. IEEE Transactions on Robotics. 2012;28(4):886-898

110 Applications of Mobile Robots

Robotic perception is related to many applications in robotics where sensory data and artificial intelligence/machine learning (AI/ML) techniques are involved. Examples of such applications are object detection, environment representation, scene understanding, human/pedestrian detection, activity recognition, semantic place classification, object modeling, among others. Robotic perception, in the scope of this chapter, encompasses the ML algorithms and techniques that empower robots to learn from sensory data and, based on learned models, to react and take decisions accordingly. The recent developments in machine learning, namely deep-learning approaches, are evident and, consequently, robotic perception systems are evolving in a way that new applications and tasks are becoming a reality. Recent advances in human-robot interaction, complex robotic tasks, intelligent reasoning, and decision-making are, at some extent, the results of the notorious evolution and success of ML algorithms. This chapter will cover recent and emerging topics and use-cases related to intelligent perception systems in robotics.

DOI: 10.5772/intechopen.79742

**Keywords:** robotic perception, machine learning, advanced robotics, artificial intelligence

#### **1. Introduction**

In robotics, perception is understood as a system that endows the robot with the ability to perceive, comprehend, and reason about the surrounding environment. The key components of a perception system are essentially sensory data processing, data representation (environment modeling), and ML-based algorithms, as illustrated in **Figure 1**. Since *strong AI* is still far from being achieved in real-world robotics applications, this chapter is about *weak AI*, i.e., standard machine learning approaches [1].

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

environments: indoors or outdoors. Therefore, different assumptions can be incorporated in the mapping (representation) and perception systems considering indoor or outdoor environments. Moreover, the sensors used are different depending on the environment, and therefore, the sensory data to be processed by a perception system will not be the same for indoors and outdoors scenarios. An example to clarify the differences and challenges between a mobile robot navigating in an indoor versus outdoor environment is the ground, or terrain, where the robot operates. Most of indoor robots assume that the ground is regular and flat which, in some manner, facilitates the environment representation models; on the other hand, for field (outdoors) robots, the terrain is quite often far from being regular and, as consequence, the environment modeling is itself a challenge and, without a proper representation, the subsequent perception tasks are negatively affected. Moreover, in outdoors, robotic perception has

Intelligent Robotic Perception Systems http://dx.doi.org/10.5772/intechopen.79742 113

Similar scenario-specific differences exist in virtually all use-cases of robotic vision, as exemplified by the 2016 Amazon Picking Challenge participants' survey [18], requiring complex yet robust solutions, and therefore considered one of the most difficult tasks in the pick-and-place application domain. Moreover, one of the participating teams from 2016 benchmarked a pose estimation method on a warehouse logistics dataset, and found large variations in performance depending on clutter level and object type [2]. Thus, perception systems currently require expert knowledge in order to select, adapt, extend, and fine-tune the various employed components. Apart from the increased training data sizes and robustness, the end-to-end training aspect of deep-learning (DL) approaches made the development of perception systems easier and more accessible for newcomers, as one can obtain the desired results directly from raw data in many cases, by providing a large number of training examples. The method selection often boils down to obtaining the latest pretrained network from an online repository and fine-tuning it to the problem at hand, hiding all the traditional feature detection, description, filtering, matching, optimization steps behind a relatively unified framework. Unfortunately, at the moment an off-the-shelf DL solution for every problem does not exist, or at least no usable pretrained network, making the need for huge amounts of training data apparent. Therefore, large datasets are a valuable asset for modern AI/ML. A large number of datasets exist for perception tasks as well, with a survey of RGB-D datasets presented by Firman [5] (up-to-date list available online: http://www.michaelfirman.co.uk/RGBDdatasets/), and even tools for synthetically generating sensor-based datasets, e.g., the work presented by Handa et al. [4] which is available online: http://robotvault.bitbucket.org/. However, the danger is to overfit to such benchmarks, as the deployment environment of mobile robots is almost sure to differ from the one used in teaching the robot to perceive and understand the surrounding environment. Thus, the suggestions formulated by Wagstaff [19] still hold true today and

As pointed out recently by Sünderhauf et al. [17], robotic perception (also designated robotic vision in [17]) differs from traditional computer vision perception in the sense that, in robotics, the outputs of a perception system will result in decisions and actions in the real world. Therefore, perception is a very important part of a complex, embodied, active, and goal-driven robotic system. As exemplified by Sünderhauf et al. [17], robotic perception has to translate images (or scans, or point-clouds) into actions, whereas most computer vision applications

to deal with weather conditions and variations in light intensities and spectra.

should be taken to heart by researchers and practitioners.

take images and translate the outputs into information.

**Figure 1.** Key modules of a typical robotic perception system: sensory data processing (focusing here on visual and range perception); data representations specific for the tasks at hand; algorithms for data analysis and interpretation (using AI/ML methods); and planning and execution of actions for robot-environment interaction.

Robotic perception is crucial for a robot to make decisions, plan, and operate in real-world environments, by means of numerous functionalities and operations from occupancy grid mapping to object detection. Some examples of robotic perception subareas, including autonomous robot-vehicles, are obstacle detection [2, 3], object recognition [4, 5], semantic place classification [6, 7], 3D environment representation [8], gesture and voice recognition [9], activity classification [10], terrain classification [11], road detection [12], vehicle detection [13], pedestrian detection [14], object tracking [3], human detection [15], and environment change detection [16].

Nowadays, most of robotic perception systems use machine learning (ML) techniques, ranging from classical to deep-learning approaches [17]. Machine learning for robotic perception can be in the form of unsupervised learning, or supervised classifiers using handcrafted features, or deep-learning neural networks (e.g., convolutional neural network (CNN)), or even a combination of multiple methods.

Regardless of the ML approach considered, data from sensor(s) are the key ingredient in robotic perception. Data can come from a single or multiple sensors, usually mounted onboard the robot, but can also come from the infrastructure or from another robot (e.g., cameras mounted on UAVs flying nearby). In multiple-sensors perception, either the same modality or multimodal, an efficient approach is usually necessary to combine and process data from the sensors before an ML method can be employed. Data alignment and calibration steps are necessary depending on the nature of the problem and the type of sensors used.

Sensor-based environment representation/mapping is a very important part of a robotic perception system. Mapping here encompasses both the acquisition of a metric model and its semantic interpretation, and is therefore a synonym of environment/scene representation. This semantic mapping process uses ML at various levels, e.g., reasoning on volumetric occupancy and occlusions, or identifying, describing, and matching optimally the local regions from different time-stamps/models, i.e., not only higher level interpretations. However, in the majority of applications, the primary role of environment mapping is to model data from exteroceptive sensors, mounted onboard the robot, in order to enable reasoning and inference regarding the real-world environment where the robot operates.

Robot perception functions, like localization and navigation, are dependent on the environment where the robot operates. Essentially, a robot is designed to operate in two categories of environments: indoors or outdoors. Therefore, different assumptions can be incorporated in the mapping (representation) and perception systems considering indoor or outdoor environments. Moreover, the sensors used are different depending on the environment, and therefore, the sensory data to be processed by a perception system will not be the same for indoors and outdoors scenarios. An example to clarify the differences and challenges between a mobile robot navigating in an indoor versus outdoor environment is the ground, or terrain, where the robot operates. Most of indoor robots assume that the ground is regular and flat which, in some manner, facilitates the environment representation models; on the other hand, for field (outdoors) robots, the terrain is quite often far from being regular and, as consequence, the environment modeling is itself a challenge and, without a proper representation, the subsequent perception tasks are negatively affected. Moreover, in outdoors, robotic perception has to deal with weather conditions and variations in light intensities and spectra.

Similar scenario-specific differences exist in virtually all use-cases of robotic vision, as exemplified by the 2016 Amazon Picking Challenge participants' survey [18], requiring complex yet robust solutions, and therefore considered one of the most difficult tasks in the pick-and-place application domain. Moreover, one of the participating teams from 2016 benchmarked a pose estimation method on a warehouse logistics dataset, and found large variations in performance depending on clutter level and object type [2]. Thus, perception systems currently require expert knowledge in order to select, adapt, extend, and fine-tune the various employed components.

Robotic perception is crucial for a robot to make decisions, plan, and operate in real-world environments, by means of numerous functionalities and operations from occupancy grid mapping to object detection. Some examples of robotic perception subareas, including autonomous robot-vehicles, are obstacle detection [2, 3], object recognition [4, 5], semantic place classification [6, 7], 3D environment representation [8], gesture and voice recognition [9], activity classification [10], terrain classification [11], road detection [12], vehicle detection [13], pedestrian detection [14], object tracking [3], human detection [15], and environment change detection [16].

**Figure 1.** Key modules of a typical robotic perception system: sensory data processing (focusing here on visual and range perception); data representations specific for the tasks at hand; algorithms for data analysis and interpretation

(using AI/ML methods); and planning and execution of actions for robot-environment interaction.

Nowadays, most of robotic perception systems use machine learning (ML) techniques, ranging from classical to deep-learning approaches [17]. Machine learning for robotic perception can be in the form of unsupervised learning, or supervised classifiers using handcrafted features, or deep-learning neural networks (e.g., convolutional neural network (CNN)), or even

Regardless of the ML approach considered, data from sensor(s) are the key ingredient in robotic perception. Data can come from a single or multiple sensors, usually mounted onboard the robot, but can also come from the infrastructure or from another robot (e.g., cameras mounted on UAVs flying nearby). In multiple-sensors perception, either the same modality or multimodal, an efficient approach is usually necessary to combine and process data from the sensors before an ML method can be employed. Data alignment and calibration steps are necessary depending on the nature of the problem and the type of sensors used.

Sensor-based environment representation/mapping is a very important part of a robotic perception system. Mapping here encompasses both the acquisition of a metric model and its semantic interpretation, and is therefore a synonym of environment/scene representation. This semantic mapping process uses ML at various levels, e.g., reasoning on volumetric occupancy and occlusions, or identifying, describing, and matching optimally the local regions from different time-stamps/models, i.e., not only higher level interpretations. However, in the majority of applications, the primary role of environment mapping is to model data from exteroceptive sensors, mounted onboard the robot, in order to enable reasoning and inference

Robot perception functions, like localization and navigation, are dependent on the environment where the robot operates. Essentially, a robot is designed to operate in two categories of

regarding the real-world environment where the robot operates.

a combination of multiple methods.

112 Applications of Mobile Robots

Apart from the increased training data sizes and robustness, the end-to-end training aspect of deep-learning (DL) approaches made the development of perception systems easier and more accessible for newcomers, as one can obtain the desired results directly from raw data in many cases, by providing a large number of training examples. The method selection often boils down to obtaining the latest pretrained network from an online repository and fine-tuning it to the problem at hand, hiding all the traditional feature detection, description, filtering, matching, optimization steps behind a relatively unified framework. Unfortunately, at the moment an off-the-shelf DL solution for every problem does not exist, or at least no usable pretrained network, making the need for huge amounts of training data apparent. Therefore, large datasets are a valuable asset for modern AI/ML. A large number of datasets exist for perception tasks as well, with a survey of RGB-D datasets presented by Firman [5] (up-to-date list available online: http://www.michaelfirman.co.uk/RGBDdatasets/), and even tools for synthetically generating sensor-based datasets, e.g., the work presented by Handa et al. [4] which is available online: http://robotvault.bitbucket.org/. However, the danger is to overfit to such benchmarks, as the deployment environment of mobile robots is almost sure to differ from the one used in teaching the robot to perceive and understand the surrounding environment. Thus, the suggestions formulated by Wagstaff [19] still hold true today and should be taken to heart by researchers and practitioners.

As pointed out recently by Sünderhauf et al. [17], robotic perception (also designated robotic vision in [17]) differs from traditional computer vision perception in the sense that, in robotics, the outputs of a perception system will result in decisions and actions in the real world. Therefore, perception is a very important part of a complex, embodied, active, and goal-driven robotic system. As exemplified by Sünderhauf et al. [17], robotic perception has to translate images (or scans, or point-clouds) into actions, whereas most computer vision applications take images and translate the outputs into information.

#### **2. Environment representation**

Among the numerous approaches used in environment representation for mobile robotics, and for autonomous robotic-vehicles, the most influential approach is the occupancy grid mapping [20]. This 2D mapping is still used in many mobile platforms due to its efficiency, probabilistic framework, and fast implementation. Although many approaches use 2D-based representations to model the real world, presently 2.5D and 3D representation models are becoming more common. The main reasons for using higher dimensional representations are essentially twofold: (1) robots are demanded to navigate and make decisions in higher complex environments where 2D representations are insufficient; (2) current 3D sensor technologies are affordable and reliable, and therefore 3D environment representations became attainable. Moreover, the recent advances in software tools, like ROS and PCL, and also the advent of methods like Octomaps, developed by Hornung et al. [21], have been contributing to the increase in 3D-like environment representations.

and outputting a mesh containing the semantic segmentation of the input data. However, the main limitation in [34] is that the approach requires knowledge of the positions from which

Intelligent Robotic Perception Systems http://dx.doi.org/10.5772/intechopen.79742 115

The recent work presented by Brucker et al. [7] builds on the segmentation of Ambrus et al. [31, 32] and explores ways of fusing different types of information, such as presence of objects and cues of the types of rooms to obtain a semantic segmentation of the environment. The aim of the work presented by Brucker et al. [7] is to obtain an intuitive and human-like labeling of the environment while at the same time preserving as many of the semantic features as possible. Also, Brucker et al. [7] use a conditional random field (CRF) or the fusion of various

Processing sensory data and storing it in a representation of the environment (i.e., a map of the environment) has been and continues to be an active area in robotics research, including autonomous driving system (or autonomous robotic-vehicles). The approaches covered range from metric representations (2D or 3D) to higher semantic or topological maps, and all serve specific purposes key to the successful operation of a mobile robot, such as localization, navigation, object detection, manipulation, etc. Moreover, the ability to construct a geometrically accurate map further annotated with semantic information also can be used in other applications such as building management or architecture, or can be further fed back into a robotic system, increasing the awareness of its surroundings and thus improving its ability to perform certain tasks in human-populated environments (e.g., finding a cup is more likely to be successful if the robot knows a priori which room is the kitchen and how to get there).

Once a robot is (self) localized, it can proceed with the execution of its task. In the case of autonomous mobile manipulators, this involves localizing the objects of interest in the operating environment and grasping them. In a typical setup, the robot navigates to the region of interest, observes the current scene to build a 3D map for collision-free grasp planning and for localizing target objects. The target could be a table or container where something has to be put down, or an object to be picked up. Especially in the latter case, estimating all 6 degrees of freedom of an object is necessary. Subsequently, a motion and a grasp are computed and executed. There are cases where a tighter integration of perception and manipulation is required, e.g., for high-precision manipulation, where approaches like visual servoing are employed. However, in every application, there is a potential improvement for treating perception and manipulation together. Perception and manipulation are complementary ways to understand and interact with the environment and according to the common coding theory, as developed and presented by Sperry [35], they are also inextricably linked in the brain. The importance of a tight link between perception and action for artificial agents has been recognized by Turing [36], who suggested to equip computers "with the best sense organs that money can buy" and let them

learn from gathered experiences until they pass his famous test as described in [37].

heterogeneous data sources and inference is done using Gibbs sampling technique.

**3. Artificial intelligence and machine learning applied on** 

**robotics perception**

the environment was scanned when the input data were collected.

The advent and proliferation of RGBD sensors has enabled the construction of larger and evermore detailed 3D maps. In addition, considerable effort has been made in the semantic labeling of these maps, at pixel and voxels levels. Most of the relevant approaches can be split into two main trends: methods designed for online and those designed for offline use. Online methods process data as it is being acquired by the mobile robot, and generate a semantic map incrementally. These methods are usually coupled with a SLAM framework, which ensures the geometric consistency of the map. Building maps of the environment is a crucial part of any robotic system and arguably one of the most researched areas in robotics. Early work coupled mapping with localization as part of the simultaneous localization and mapping (SLAM) problem [22, 23]. More recent work has focused on dealing with or incorporating time-dependencies (short or long term) into the underlying structure, using either grid maps as described in [8, 24], pose-graph representations in [25], and normal distribution transform (NDT) [16, 26].

As presented by Hermans et al. [27], RGBD data are processed by a random forest-based classifier and predict semantic labels; these labels are further regularized through the conditional random field (CRF) method proposed by Krahenbuhl and Koltun [28]. Similarly, McCormac et al. [29] use the elastic fusion SLAM algorithm proposed by Whelan et al. [30] to fuse CNN predictions about the scene in a geometrically consistent map. In the work of Sünderhauf et al. [6], a CNN is used to incrementally build a semantic map, with the aim of extending the number of classes supported by the CNN by complementing it with a series of one-vs-all classifiers which can be trained online. A number of semantic mapping approaches are designed to operate offline, taking as input a complete map of the environment. In the methods described by Ambrus et al. [31, 32] and Armeni et al. [33], large-scale point clouds of indoor buildings are processed, and then, after segmenting the input data, the method's outputs are in the form of a set of "rooms." Ambrus et al. [31, 32] use a 2D cell-complex graph-cut approach to compute the segmentation with the main limitation that only single floor buildings can be processed, while Armeni et al. [33] process multifloor structures by detecting the spaces between the walls, ceilings, etc., with the limitation that the building walls have to be axis-aligned (i.e., the Manhattan world assumption). Similarly, in the work proposed by Mura et al. [34], a large point cloud of an indoor structure is processed by making use of a 3D cell-complex structure and outputting a mesh containing the semantic segmentation of the input data. However, the main limitation in [34] is that the approach requires knowledge of the positions from which the environment was scanned when the input data were collected.

**2. Environment representation**

114 Applications of Mobile Robots

to the increase in 3D-like environment representations.

Among the numerous approaches used in environment representation for mobile robotics, and for autonomous robotic-vehicles, the most influential approach is the occupancy grid mapping [20]. This 2D mapping is still used in many mobile platforms due to its efficiency, probabilistic framework, and fast implementation. Although many approaches use 2D-based representations to model the real world, presently 2.5D and 3D representation models are becoming more common. The main reasons for using higher dimensional representations are essentially twofold: (1) robots are demanded to navigate and make decisions in higher complex environments where 2D representations are insufficient; (2) current 3D sensor technologies are affordable and reliable, and therefore 3D environment representations became attainable. Moreover, the recent advances in software tools, like ROS and PCL, and also the advent of methods like Octomaps, developed by Hornung et al. [21], have been contributing

The advent and proliferation of RGBD sensors has enabled the construction of larger and evermore detailed 3D maps. In addition, considerable effort has been made in the semantic labeling of these maps, at pixel and voxels levels. Most of the relevant approaches can be split into two main trends: methods designed for online and those designed for offline use. Online methods process data as it is being acquired by the mobile robot, and generate a semantic map incrementally. These methods are usually coupled with a SLAM framework, which ensures the geometric consistency of the map. Building maps of the environment is a crucial part of any robotic system and arguably one of the most researched areas in robotics. Early work coupled mapping with localization as part of the simultaneous localization and mapping (SLAM) problem [22, 23]. More recent work has focused on dealing with or incorporating time-dependencies (short or long term) into the underlying structure, using either grid maps as described in [8, 24],

pose-graph representations in [25], and normal distribution transform (NDT) [16, 26].

As presented by Hermans et al. [27], RGBD data are processed by a random forest-based classifier and predict semantic labels; these labels are further regularized through the conditional random field (CRF) method proposed by Krahenbuhl and Koltun [28]. Similarly, McCormac et al. [29] use the elastic fusion SLAM algorithm proposed by Whelan et al. [30] to fuse CNN predictions about the scene in a geometrically consistent map. In the work of Sünderhauf et al. [6], a CNN is used to incrementally build a semantic map, with the aim of extending the number of classes supported by the CNN by complementing it with a series of one-vs-all classifiers which can be trained online. A number of semantic mapping approaches are designed to operate offline, taking as input a complete map of the environment. In the methods described by Ambrus et al. [31, 32] and Armeni et al. [33], large-scale point clouds of indoor buildings are processed, and then, after segmenting the input data, the method's outputs are in the form of a set of "rooms." Ambrus et al. [31, 32] use a 2D cell-complex graph-cut approach to compute the segmentation with the main limitation that only single floor buildings can be processed, while Armeni et al. [33] process multifloor structures by detecting the spaces between the walls, ceilings, etc., with the limitation that the building walls have to be axis-aligned (i.e., the Manhattan world assumption). Similarly, in the work proposed by Mura et al. [34], a large point cloud of an indoor structure is processed by making use of a 3D cell-complex structure The recent work presented by Brucker et al. [7] builds on the segmentation of Ambrus et al. [31, 32] and explores ways of fusing different types of information, such as presence of objects and cues of the types of rooms to obtain a semantic segmentation of the environment. The aim of the work presented by Brucker et al. [7] is to obtain an intuitive and human-like labeling of the environment while at the same time preserving as many of the semantic features as possible. Also, Brucker et al. [7] use a conditional random field (CRF) or the fusion of various heterogeneous data sources and inference is done using Gibbs sampling technique.

Processing sensory data and storing it in a representation of the environment (i.e., a map of the environment) has been and continues to be an active area in robotics research, including autonomous driving system (or autonomous robotic-vehicles). The approaches covered range from metric representations (2D or 3D) to higher semantic or topological maps, and all serve specific purposes key to the successful operation of a mobile robot, such as localization, navigation, object detection, manipulation, etc. Moreover, the ability to construct a geometrically accurate map further annotated with semantic information also can be used in other applications such as building management or architecture, or can be further fed back into a robotic system, increasing the awareness of its surroundings and thus improving its ability to perform certain tasks in human-populated environments (e.g., finding a cup is more likely to be successful if the robot knows a priori which room is the kitchen and how to get there).

### **3. Artificial intelligence and machine learning applied on robotics perception**

Once a robot is (self) localized, it can proceed with the execution of its task. In the case of autonomous mobile manipulators, this involves localizing the objects of interest in the operating environment and grasping them. In a typical setup, the robot navigates to the region of interest, observes the current scene to build a 3D map for collision-free grasp planning and for localizing target objects. The target could be a table or container where something has to be put down, or an object to be picked up. Especially in the latter case, estimating all 6 degrees of freedom of an object is necessary. Subsequently, a motion and a grasp are computed and executed. There are cases where a tighter integration of perception and manipulation is required, e.g., for high-precision manipulation, where approaches like visual servoing are employed. However, in every application, there is a potential improvement for treating perception and manipulation together.

Perception and manipulation are complementary ways to understand and interact with the environment and according to the common coding theory, as developed and presented by Sperry [35], they are also inextricably linked in the brain. The importance of a tight link between perception and action for artificial agents has been recognized by Turing [36], who suggested to equip computers "with the best sense organs that money can buy" and let them learn from gathered experiences until they pass his famous test as described in [37].

The argument for embodied learning and grounding of new information evolved, considering the works of Steels and Brooks [38] and Vernon [39], and more recently in [40], robot perception involves planning and interactive segmentation. In this regard, perception and action reciprocally inform each other, in order to obtain the best results for locating objects. In this context, the localization problem involves segmenting objects, but also knowing their position and orientation relative to the robot in order to facilitate manipulation. The problem of object pose estimation, an important prerequisite for model-based robotic grasping, uses in most of the cases precomputed grasp points as described by Ferrari and Canny [41]. We can categorize this topic in either template/descriptor-based approaches or alternatively local feature/patch-based approaches. In both cases, an ever-recurring approach is that bottom-up data-driven hypothesis generation is followed and verified by top-down concept-driven models. Such mechanisms are assumed, as addressed by Frisby and Stone [42], to be like our human vision system.

Challenges (2004, 2005, and 2007) and the European ELROB challenges (since 2006), and more recently, it has regained considerable interest from automotive and robotics industries and academia. Research in self-driving cars, also referred as autonomous robot-cars, is closely related to mobile robotics and many important works in this field have been published in well-known conferences and journals devoted to robotics. Autonomous driving systems (ADS) comprise, basically, perception (including sensor-fusion and environment modeling/ representation), localization, and navigation (path planning, trajectory following, control) and, more recently, cooperation (V2X-based communication technologies). However, the cornerstone of ADS is the perception system because it is involved in most of the essential and necessary tasks for safe driving such as the "segmentation," detection/recognition, of: road, lane-markings, pedestrians, and other vulnerable road users (e.g., cyclists), other vehicles, traffic signals, crosswalks, and the numerous other types of objects and obstacles that can be found on the roads. In addition to the sensors (e.g., cameras, LIDAR, Radar, "new" solid-state LiDAR technology) and the models used in ADS, the common denominator in a perception system consists of AI/ML algorithms, where deep learning is the leading

Intelligent Robotic Perception Systems http://dx.doi.org/10.5772/intechopen.79742 117

One of current trends in autonomous vehicles and robotics is the promising idea of incorporating cooperative information, from connected environment/infrastructure, into the decision loop of the robotic perception system. The rationale is to improve robustness and safety by providing complementary information to the perception system, for example: the position and identification of a given object or obstacle on the road could be reported (e.g., broadcasted through a communication network) in advance to an autonomous car, moments before the

The EU FP7 Strands project [52] is formed by a consortium of six universities and two industrial partners. The aim of the project is to develop the next generation of intelligent mobile robots, capable of operating alongside humans for extended periods of time. While research into mobile robotic technology has been very active over the last few decades, robotic systems that can operate robustly, for extended periods of time, in human-populated environments remain a rarity. Strands aims to fill this gap and to provide robots that are intelligent, robust, and can provide useful functions in real-world security and care scenarios. Importantly, the extended operation times imply that the robotic systems developed have to be able to cope with an ever-increasing amount of data, as well as to be able to deal with the complex and

**Figure 3** shows a high level overview of the Strands system (with more details in [52]): the mobile robot navigates autonomously between a number of predefined waypoints. A task scheduling mechanism dictates when the robot should visit which waypoints, depending on the tasks the robot has to accomplish on any given day. The perception system consists, at the lowest level, of a module which builds local metric maps at the waypoints visited by the robot.

technique for semantic segmentation and object detection [50].

object/obstacle are within the onboard sensor's field/range of view.

**4. Case studies**

**4.1. The Strands project**

unstructured real world (**Figure 2**).

The approaches presented in ([43–45] make use of color histograms, color gradients, depth or normal orientations from discrete object views, i.e., they are examples of vision-/camerabased perception for robots. Vision-based perception systems typically suffer from occlusions, aspect ratio influence, and from problems arising due to the discretization of the 3D or 6D search space. Conversely, in the works of [46–48], they predict the object pose through voting or a PnP algorithm [49]. The performance usually decreases if the considered object lacks texture and if the background is heavily cluttered. In the works listed above, learning algorithms based on classical ML methods and deep-learning (e.g., CNN) have been employed.

The importance of mobile manipulation and perception areas has been signaled by the (not only academic) interest spurred by events like the Amazon Robotics (formerly Picking) Challenge and the workshop series at the recent major computer vision conferences associated with the SIXD Challenge (http://cmp.felk.cvut.cz/sixd/workshop\_2018/). However, current solutions are either heavily tailored to a specific application, requiring specific engineering during deployment, or their generality makes them too slow or imprecise to fulfill the tight time-constraints of industrial applications. While deep learning holds the potential to both improve accuracy (i.e., classification or recognition performance) and also to increase execution speed, more work on transfer learning, in the sense of generalization improvement, is required to apply models learned in real-world and also in unseen (new) environment. Domain adaptation and domain randomization (i.e., image augmentations) seem to be important directions to pursue, and should be explored not only for vision/camera cases, but also for LiDAR-based perception cases.

Usually, in traditional mobile robot manipulation use-cases, the navigation and manipulation capabilities of a robot can be exploited to let the robot gather data about objects autonomously. This can involve, for instance, observing an object of interest from multiple viewpoints in order to allow a better object model estimation, or even in-hand modeling. In the case of perception for mobile robots and autonomous (robot) vehicles, such options are not available; thus, its perception systems have to be trained offline. However, besides AI/ML-based algorithms and higher level perception, for autonomous driving applications, environment representation (including multisensor fusion) is of primary concern [50, 51].

The development of advanced perception for (full) autonomous driving has been a subject of interest since the 1980s, having a period of strong development due to the DARPA Challenges (2004, 2005, and 2007) and the European ELROB challenges (since 2006), and more recently, it has regained considerable interest from automotive and robotics industries and academia. Research in self-driving cars, also referred as autonomous robot-cars, is closely related to mobile robotics and many important works in this field have been published in well-known conferences and journals devoted to robotics. Autonomous driving systems (ADS) comprise, basically, perception (including sensor-fusion and environment modeling/ representation), localization, and navigation (path planning, trajectory following, control) and, more recently, cooperation (V2X-based communication technologies). However, the cornerstone of ADS is the perception system because it is involved in most of the essential and necessary tasks for safe driving such as the "segmentation," detection/recognition, of: road, lane-markings, pedestrians, and other vulnerable road users (e.g., cyclists), other vehicles, traffic signals, crosswalks, and the numerous other types of objects and obstacles that can be found on the roads. In addition to the sensors (e.g., cameras, LIDAR, Radar, "new" solid-state LiDAR technology) and the models used in ADS, the common denominator in a perception system consists of AI/ML algorithms, where deep learning is the leading technique for semantic segmentation and object detection [50].

One of current trends in autonomous vehicles and robotics is the promising idea of incorporating cooperative information, from connected environment/infrastructure, into the decision loop of the robotic perception system. The rationale is to improve robustness and safety by providing complementary information to the perception system, for example: the position and identification of a given object or obstacle on the road could be reported (e.g., broadcasted through a communication network) in advance to an autonomous car, moments before the object/obstacle are within the onboard sensor's field/range of view.

#### **4. Case studies**

The argument for embodied learning and grounding of new information evolved, considering the works of Steels and Brooks [38] and Vernon [39], and more recently in [40], robot perception involves planning and interactive segmentation. In this regard, perception and action reciprocally inform each other, in order to obtain the best results for locating objects. In this context, the localization problem involves segmenting objects, but also knowing their position and orientation relative to the robot in order to facilitate manipulation. The problem of object pose estimation, an important prerequisite for model-based robotic grasping, uses in most of the cases precomputed grasp points as described by Ferrari and Canny [41]. We can categorize this topic in either template/descriptor-based approaches or alternatively local feature/patch-based approaches. In both cases, an ever-recurring approach is that bottom-up data-driven hypothesis generation is followed and verified by top-down concept-driven models. Such mechanisms are assumed, as addressed by Frisby and Stone [42], to be like our human vision system.

The approaches presented in ([43–45] make use of color histograms, color gradients, depth or normal orientations from discrete object views, i.e., they are examples of vision-/camerabased perception for robots. Vision-based perception systems typically suffer from occlusions, aspect ratio influence, and from problems arising due to the discretization of the 3D or 6D search space. Conversely, in the works of [46–48], they predict the object pose through voting or a PnP algorithm [49]. The performance usually decreases if the considered object lacks texture and if the background is heavily cluttered. In the works listed above, learning algorithms

The importance of mobile manipulation and perception areas has been signaled by the (not only academic) interest spurred by events like the Amazon Robotics (formerly Picking) Challenge and the workshop series at the recent major computer vision conferences associated with the SIXD Challenge (http://cmp.felk.cvut.cz/sixd/workshop\_2018/). However, current solutions are either heavily tailored to a specific application, requiring specific engineering during deployment, or their generality makes them too slow or imprecise to fulfill the tight time-constraints of industrial applications. While deep learning holds the potential to both improve accuracy (i.e., classification or recognition performance) and also to increase execution speed, more work on transfer learning, in the sense of generalization improvement, is required to apply models learned in real-world and also in unseen (new) environment. Domain adaptation and domain randomization (i.e., image augmentations) seem to be important directions to pursue, and should be explored not only for vision/camera cases, but also

Usually, in traditional mobile robot manipulation use-cases, the navigation and manipulation capabilities of a robot can be exploited to let the robot gather data about objects autonomously. This can involve, for instance, observing an object of interest from multiple viewpoints in order to allow a better object model estimation, or even in-hand modeling. In the case of perception for mobile robots and autonomous (robot) vehicles, such options are not available; thus, its perception systems have to be trained offline. However, besides AI/ML-based algorithms and higher level perception, for autonomous driving applications, environment

The development of advanced perception for (full) autonomous driving has been a subject of interest since the 1980s, having a period of strong development due to the DARPA

representation (including multisensor fusion) is of primary concern [50, 51].

based on classical ML methods and deep-learning (e.g., CNN) have been employed.

for LiDAR-based perception cases.

116 Applications of Mobile Robots

#### **4.1. The Strands project**

The EU FP7 Strands project [52] is formed by a consortium of six universities and two industrial partners. The aim of the project is to develop the next generation of intelligent mobile robots, capable of operating alongside humans for extended periods of time. While research into mobile robotic technology has been very active over the last few decades, robotic systems that can operate robustly, for extended periods of time, in human-populated environments remain a rarity. Strands aims to fill this gap and to provide robots that are intelligent, robust, and can provide useful functions in real-world security and care scenarios. Importantly, the extended operation times imply that the robotic systems developed have to be able to cope with an ever-increasing amount of data, as well as to be able to deal with the complex and unstructured real world (**Figure 2**).

**Figure 3** shows a high level overview of the Strands system (with more details in [52]): the mobile robot navigates autonomously between a number of predefined waypoints. A task scheduling mechanism dictates when the robot should visit which waypoints, depending on the tasks the robot has to accomplish on any given day. The perception system consists, at the lowest level, of a module which builds local metric maps at the waypoints visited by the robot.

further be used to generate a textured mesh through which a convolutional neural network can be trained which can successfully recognize the object in future observations [31, 32]. The dynamics detected in the environment can be used to detect patterns, either through spectral analysis (i.e., by applying a Fourier transform on the raw detection data), as described in [54],

Intelligent Robotic Perception Systems http://dx.doi.org/10.5772/intechopen.79742 119

In addition to the detection and modeling of objects, the Strands perception system also focuses on the detection of people. Beyer et al. [55] present a method to continuously estimate the head-pose of people, while in [15] laser and RGB-D are combined to reliably detect humans and to allow human-aware navigation approaches which make the robot more socially acceptable. Beyer et al. [56] propose a CNN-based system which uses laser scanner data to detect objects; the usefulness of the approach is demonstrated in the case scenario, where it is used to

Robust perception algorithms that can operate reliably for extended periods of time are one of the cornerstones of the Strands system. However, any algorithm deployed on the robot has to be not only robust, but also able to scale as the robot makes more observations and collects more information about the world. One of the key parts that would enable the successful operation of such a robotic system is a perception stack that is able to continuously integrate observations about the world, extract relevant parts as well as build models that understand and are able to predict what the environment will look like in the future. This spatio-temporal understanding is crucial, as it allows a mobile robot to compress the data acquired during months of autonomous operation into models that can be used to refine the robot's operation over time. Modeling periodicities in the environment and integrating them into a planning pipeline is further investigated by Fentanes et al. [57], while Santos et al. [58] build spatio-temporal models of the environment and use them for exploration through an information-theoretic approach which predicts the potential gain of observing particular

Advanced robots operating in complex and dynamic environments require intelligent perception algorithms to navigate collision-free, analyze scenes, recognize relevant objects, and manipulate them. Nowadays, the perception of mobile manipulation systems often fails if the context changes due to a variation, e.g., in the lightning conditions, the utilized objects, the manipulation area, or the environment. Then, a robotic expert is needed who needs to adjust the parameters of the perception algorithm and the utilized sensor or even select a better method or sensor. Thus, a high-level cognitive ability that is required for operating alongside humans is to continuously improve performance based on introspection. This adaptability to changing situations requires different aspects of machine learning, e.g., storing experiences for life-long learning, generating annotated datasets for supervised learning through user interaction, Bayesian optimization to avoid brute-force search in high-dimensional data, and a unified

The RobDREAM consortium automated and integrated different aspects of these. Specifically, in the EU's H2020 RobDREAM project, a mobile manipulator was used to showcase the

representation of data and meta-data to facilitate knowledge transfer.

or as part of a multitarget tracking system based on a Rao-Blackwellized particle filter.

detect wheelchairs and walkers.

areas of the world at different points in time.

**4.2. The RobDREAM project**

**Figure 2.** The Strands project (image from http://strands.acin.tuwien.ac.at/).

These local maps are updated over time, as the robot revisits the same locations in the environment, and they are further used to segment out the dynamic objects from the static scene. The dynamic segmentations are used as cues for higher level behaviors, such as triggering a data acquisition and object modeling step, whereby the robot navigates around the detected object to collect additional data which are fused into a canonical model of the object [53]. The data can

**Figure 3.** The Strands system—Overview.

further be used to generate a textured mesh through which a convolutional neural network can be trained which can successfully recognize the object in future observations [31, 32]. The dynamics detected in the environment can be used to detect patterns, either through spectral analysis (i.e., by applying a Fourier transform on the raw detection data), as described in [54], or as part of a multitarget tracking system based on a Rao-Blackwellized particle filter.

In addition to the detection and modeling of objects, the Strands perception system also focuses on the detection of people. Beyer et al. [55] present a method to continuously estimate the head-pose of people, while in [15] laser and RGB-D are combined to reliably detect humans and to allow human-aware navigation approaches which make the robot more socially acceptable. Beyer et al. [56] propose a CNN-based system which uses laser scanner data to detect objects; the usefulness of the approach is demonstrated in the case scenario, where it is used to detect wheelchairs and walkers.

Robust perception algorithms that can operate reliably for extended periods of time are one of the cornerstones of the Strands system. However, any algorithm deployed on the robot has to be not only robust, but also able to scale as the robot makes more observations and collects more information about the world. One of the key parts that would enable the successful operation of such a robotic system is a perception stack that is able to continuously integrate observations about the world, extract relevant parts as well as build models that understand and are able to predict what the environment will look like in the future. This spatio-temporal understanding is crucial, as it allows a mobile robot to compress the data acquired during months of autonomous operation into models that can be used to refine the robot's operation over time. Modeling periodicities in the environment and integrating them into a planning pipeline is further investigated by Fentanes et al. [57], while Santos et al. [58] build spatio-temporal models of the environment and use them for exploration through an information-theoretic approach which predicts the potential gain of observing particular areas of the world at different points in time.

#### **4.2. The RobDREAM project**

**Figure 3.** The Strands system—Overview.

These local maps are updated over time, as the robot revisits the same locations in the environment, and they are further used to segment out the dynamic objects from the static scene. The dynamic segmentations are used as cues for higher level behaviors, such as triggering a data acquisition and object modeling step, whereby the robot navigates around the detected object to collect additional data which are fused into a canonical model of the object [53]. The data can

**Figure 2.** The Strands project (image from http://strands.acin.tuwien.ac.at/).

118 Applications of Mobile Robots

Advanced robots operating in complex and dynamic environments require intelligent perception algorithms to navigate collision-free, analyze scenes, recognize relevant objects, and manipulate them. Nowadays, the perception of mobile manipulation systems often fails if the context changes due to a variation, e.g., in the lightning conditions, the utilized objects, the manipulation area, or the environment. Then, a robotic expert is needed who needs to adjust the parameters of the perception algorithm and the utilized sensor or even select a better method or sensor. Thus, a high-level cognitive ability that is required for operating alongside humans is to continuously improve performance based on introspection. This adaptability to changing situations requires different aspects of machine learning, e.g., storing experiences for life-long learning, generating annotated datasets for supervised learning through user interaction, Bayesian optimization to avoid brute-force search in high-dimensional data, and a unified representation of data and meta-data to facilitate knowledge transfer.

The RobDREAM consortium automated and integrated different aspects of these. Specifically, in the EU's H2020 RobDREAM project, a mobile manipulator was used to showcase the

Since around 80% of passenger traffic at different hubs, including Schiphol in Amsterdam, is comprised of passengers who are transferring from one flight to the other, KLM is interested in an efficient management of their movements. For example, when transfer times are short, and finding one's way in a big airport is difficult due to language and alphabet barriers, people are at risk to losing their connection. In such, and similar cases, robotic assistants that can be deployed and booked flexibly can possibly help alleviate some of the problem. This use-case was explored by the SPENCER demonstrator for smart passengers' flow management and mobile information provider, but similar solutions are required in other domains

Intelligent Robotic Perception Systems http://dx.doi.org/10.5772/intechopen.79742 121

**Figure 5.** Concept and results of the SPENCER project (images from http://www.spencer.eu/).

as well (**Figure 5**).

**Figure 4.** Schematics of the RobDREAM approach (image based on deliverables of http://robdream.eu/).

intuitive programming and simplified setup of robotic applications enabled by automatically tuning task execution pipelines according to user-defined performance criteria.

As illustrated in **Figure 4**, this was achieved by a semantically annotated logging of perceptual episodic memories that can be queried intuitively in order to analyze the performance of the system in different contexts. Then, a ground truth annotation tool can be used by the user to mark satisfying results, or correct unsatisfying ones, where the suggestions and interactive capabilities of the system reduced the cognitive load of this often complicated task (especially when it comes to 6 DoF pose annotations), as shown in user studies involving computer vision expert and nonexpert users alike.

These annotations are then used by a Bayesian optimization framework to tune the offthe-shelf pipeline to the specific scenarios the robot encounters, thereby incrementally improving the performance of the system. The project did not focus only on perception, but on other key technologies for mobile manipulation as well. Bayesian optimization and other techniques were used to adapt the navigation, manipulation, and grasping capabilities independently of each other and the perception ones. However, the combinatorial complexity of the joint parameter space of all the involved steps was too much even for such intelligent meta-learners. The final industrially relevant use-case demo featured the kitting and mounting of electric cabinet board elements, for which a pose-annotated database was built using two RBD-D cameras and released to the public (http://www.dlr.de/ rm/thr-dataset).

#### **4.3. The SPENCER project**

When deploying robots in scenarios where they need to share the environment and interact with a large number of people, it is increasingly important that their functionalities are "socially aware." This means that they respect the personal space (and also privacy) of encountered persons, does not navigate s.t. to cut up cues or groups, etc. Such functionalities go beyond the usual focus of robotics research groups, while academics focusing on user experience typically do not have the means to develop radically new robots. However, the EU's FP7 program funded such an interdisciplinary project, called SPENCER, driven by an end-user in the aviation industry.

Since around 80% of passenger traffic at different hubs, including Schiphol in Amsterdam, is comprised of passengers who are transferring from one flight to the other, KLM is interested in an efficient management of their movements. For example, when transfer times are short, and finding one's way in a big airport is difficult due to language and alphabet barriers, people are at risk to losing their connection. In such, and similar cases, robotic assistants that can be deployed and booked flexibly can possibly help alleviate some of the problem. This use-case was explored by the SPENCER demonstrator for smart passengers' flow management and mobile information provider, but similar solutions are required in other domains as well (**Figure 5**).

intuitive programming and simplified setup of robotic applications enabled by automatically

As illustrated in **Figure 4**, this was achieved by a semantically annotated logging of perceptual episodic memories that can be queried intuitively in order to analyze the performance of the system in different contexts. Then, a ground truth annotation tool can be used by the user to mark satisfying results, or correct unsatisfying ones, where the suggestions and interactive capabilities of the system reduced the cognitive load of this often complicated task (especially when it comes to 6 DoF pose annotations), as shown in user studies involving computer

These annotations are then used by a Bayesian optimization framework to tune the offthe-shelf pipeline to the specific scenarios the robot encounters, thereby incrementally improving the performance of the system. The project did not focus only on perception, but on other key technologies for mobile manipulation as well. Bayesian optimization and other techniques were used to adapt the navigation, manipulation, and grasping capabilities independently of each other and the perception ones. However, the combinatorial complexity of the joint parameter space of all the involved steps was too much even for such intelligent meta-learners. The final industrially relevant use-case demo featured the kitting and mounting of electric cabinet board elements, for which a pose-annotated database was built using two RBD-D cameras and released to the public (http://www.dlr.de/

When deploying robots in scenarios where they need to share the environment and interact with a large number of people, it is increasingly important that their functionalities are "socially aware." This means that they respect the personal space (and also privacy) of encountered persons, does not navigate s.t. to cut up cues or groups, etc. Such functionalities go beyond the usual focus of robotics research groups, while academics focusing on user experience typically do not have the means to develop radically new robots. However, the EU's FP7 program funded such an interdisciplinary project, called SPENCER, driven by an

tuning task execution pipelines according to user-defined performance criteria.

**Figure 4.** Schematics of the RobDREAM approach (image based on deliverables of http://robdream.eu/).

vision expert and nonexpert users alike.

rm/thr-dataset).

120 Applications of Mobile Robots

**4.3. The SPENCER project**

end-user in the aviation industry.

**Figure 5.** Concept and results of the SPENCER project (images from http://www.spencer.eu/).

The SPENCER consortium integrated the developed technologies onto a robot platform whose task consists in picking up short-transfer time passenger groups at their gate of arrival, identifying them with an onboard boarding pass reader, guiding them to the Schengen barrier and instructing them to use the priority track [59]. Additionally, the platform was equipped with a KLM information kiosk and provides services to passengers in need of help.

**5. Conclusions and remarks**

factory automation.

**Author details**

**References**

Cristiano Premebida<sup>1</sup>

So just how capable is current perception and AI, and how close did/can it get to human-level performance? Szeliski [60] in his introductory book to computer vision argued that traditional vision struggled to reach the performance of a 2-year old child, but today's CNNs reach super-human classification performance on restricted domains (e.g., in the ImageNet Large

Intelligent Robotic Perception Systems http://dx.doi.org/10.5772/intechopen.79742 123

The recent surge and interest in deep-learning methods for perception has greatly improved performance in a variety of tasks such as object detection, recognition, semantic segmentation, etc. One of the main reasons for these advancements is that working on perception systems lends itself easily to offline experimentation on publicly available datasets, and comparison to

Machine learning (ML) and deep learning (DL), the latter has been one of the most used keywords in some conferences in robotics recently, are consolidated topics embraced by the robotics community nowadays. While one can interpret the filters of CNNs as Gabor filters and assume to be analogous to functions of the visual cortex, currently, deep learning is a purely nonsymbolic approach to AI/ML, and thus not expected to produce "strong" AI/ML. However, even at the current level, its usefulness is undeniable, and perhaps, the most eloquent example comes from the world of autonomous driving which brings together the robotics and the computer vision community. A number of other roboticsrelated products are starting to be commercially available for increasingly complex tasks such as visual question and answering systems, video captioning and activity recognition, large-scale human detection and tracking in videos, or anomaly detection in images for

and Zoltan-Csaba Marton<sup>3</sup>

[1] Russell SJ, Norvig P. Artificial Intelligence: A Modern Approach. 2nd ed. Upper Saddle

Scale Visual Recognition Challenge: http://www.image-net.org/challenges/LSVRC/).

other methods via standard benchmarks and competitions.

\*, Rares Ambrus<sup>2</sup>

1 Institute of Systems and Robotics (ISR-UC), Coimbra, Portugal

River, New Jersey: Prentice Hall, ISBN 0-13-790395-2; 2003

\*Address all correspondence to: cpremebida@isr.uc.pt

2 Toyota Research Institute, Los Altos, California, USA

3 German Aerospace Center, Köln, Germany

In crowded environments such as airports, generating short and safe paths for mobile robots is still difficult. Thus, social scene understanding and long-term prediction of human motion in crowds is not sufficiently solved but highly relevant for all robots that need to quickly navigate in human environments, possibly under temporal constraints. Social scene understanding means, in part, that a reliable tracking and prediction of people's motion with low uncertainty is available, and that is particularly hard if there are too many occlusions and too many fast changes of motion direction. Classical path planning approaches often result in an overconstrained or overly cautious robot that either fails to produce a feasible and safe path in the crowd, or plans a large and suboptimal detour to avoid people in the scene.

#### **4.4. The AUTOCITS project**

The AUTOCITS (https://www.autocits.eu/) project will carry out a comprehensive assessment of cooperative systems and autonomous driving by deploying real-world Pilots, and will study and review regulations related to automated and autonomous driving. AUTOCITS, cofinanced by the European Union through the Connecting Europe Facility (CEF) Program, aims to facilitate the deployment of autonomous vehicles in European roads, and to use connected/cooperative intelligent transport systems (C-ITS) services to share information between autonomous vehicles and infrastructure, by means of V2V and V2I communication technology, to improve safety and to facilitate the coexistence of autonomous cars in real-world traffic conditions. The AUTOCITS Pilots, involving connected and autonomous vehicles (including autonomous shuttles, i.e., low-speed robot-vehicles), will be deployed in three major European cities in "the Atlantic Corridor of the European Network": Lisbon (Portugal), Madrid (Spain), and Paris (France).

A number of technologies are involved in AUTOCITS, ranging from the onboard and roadside units (OBU, RSU) to the autonomous driving systems that equip the cars. Today, the autonomous and/or automated driving technology we see on the roads belongs to the levels 3 or 4 (with respect to the SAE's levels of automation in vehicles). In AUTOCITS, the Pilot's deployment will be of level 3 to 4. In this context, it is important to say that level 5 cars (i.e., 100% self-driving or full-automated cars: the driving wheels would be unnecessary) operating in real-world roads and streets are still far from reality.

We can say that the perception system is in charge of all tasks related to object and event detection and response (OEDR). Therefore, a perception system—including of course its software modules—is responsible for sensing, understanding, and reasoning about the autonomous car's surroundings. Within a connected and cooperative environment, connected cars would leverage and complement onboard sensor data by using information from vehicular communication systems (i.e., V2X technology): information from other connected vehicles, from infrastructure, and road users (and vice-versa).

#### **5. Conclusions and remarks**

The SPENCER consortium integrated the developed technologies onto a robot platform whose task consists in picking up short-transfer time passenger groups at their gate of arrival, identifying them with an onboard boarding pass reader, guiding them to the Schengen barrier and instructing them to use the priority track [59]. Additionally, the platform was equipped

In crowded environments such as airports, generating short and safe paths for mobile robots is still difficult. Thus, social scene understanding and long-term prediction of human motion in crowds is not sufficiently solved but highly relevant for all robots that need to quickly navigate in human environments, possibly under temporal constraints. Social scene understanding means, in part, that a reliable tracking and prediction of people's motion with low uncertainty is available, and that is particularly hard if there are too many occlusions and too many fast changes of motion direction. Classical path planning approaches often result in an overconstrained or overly cautious robot that either fails to produce a feasible and safe path in the crowd, or plans a large and suboptimal detour to avoid people in the scene.

The AUTOCITS (https://www.autocits.eu/) project will carry out a comprehensive assessment of cooperative systems and autonomous driving by deploying real-world Pilots, and will study and review regulations related to automated and autonomous driving. AUTOCITS, cofinanced by the European Union through the Connecting Europe Facility (CEF) Program, aims to facilitate the deployment of autonomous vehicles in European roads, and to use connected/cooperative intelligent transport systems (C-ITS) services to share information between autonomous vehicles and infrastructure, by means of V2V and V2I communication technology, to improve safety and to facilitate the coexistence of autonomous cars in real-world traffic conditions. The AUTOCITS Pilots, involving connected and autonomous vehicles (including autonomous shuttles, i.e., low-speed robot-vehicles), will be deployed in three major European cities in "the Atlantic Corridor of the European Network": Lisbon

A number of technologies are involved in AUTOCITS, ranging from the onboard and roadside units (OBU, RSU) to the autonomous driving systems that equip the cars. Today, the autonomous and/or automated driving technology we see on the roads belongs to the levels 3 or 4 (with respect to the SAE's levels of automation in vehicles). In AUTOCITS, the Pilot's deployment will be of level 3 to 4. In this context, it is important to say that level 5 cars (i.e., 100% self-driving or full-automated cars: the driving wheels would be unnecessary) operat-

We can say that the perception system is in charge of all tasks related to object and event detection and response (OEDR). Therefore, a perception system—including of course its software modules—is responsible for sensing, understanding, and reasoning about the autonomous car's surroundings. Within a connected and cooperative environment, connected cars would leverage and complement onboard sensor data by using information from vehicular communication systems (i.e., V2X technology): information from other connected vehicles,

with a KLM information kiosk and provides services to passengers in need of help.

**4.4. The AUTOCITS project**

122 Applications of Mobile Robots

(Portugal), Madrid (Spain), and Paris (France).

ing in real-world roads and streets are still far from reality.

from infrastructure, and road users (and vice-versa).

So just how capable is current perception and AI, and how close did/can it get to human-level performance? Szeliski [60] in his introductory book to computer vision argued that traditional vision struggled to reach the performance of a 2-year old child, but today's CNNs reach super-human classification performance on restricted domains (e.g., in the ImageNet Large Scale Visual Recognition Challenge: http://www.image-net.org/challenges/LSVRC/).

The recent surge and interest in deep-learning methods for perception has greatly improved performance in a variety of tasks such as object detection, recognition, semantic segmentation, etc. One of the main reasons for these advancements is that working on perception systems lends itself easily to offline experimentation on publicly available datasets, and comparison to other methods via standard benchmarks and competitions.

Machine learning (ML) and deep learning (DL), the latter has been one of the most used keywords in some conferences in robotics recently, are consolidated topics embraced by the robotics community nowadays. While one can interpret the filters of CNNs as Gabor filters and assume to be analogous to functions of the visual cortex, currently, deep learning is a purely nonsymbolic approach to AI/ML, and thus not expected to produce "strong" AI/ML. However, even at the current level, its usefulness is undeniable, and perhaps, the most eloquent example comes from the world of autonomous driving which brings together the robotics and the computer vision community. A number of other roboticsrelated products are starting to be commercially available for increasingly complex tasks such as visual question and answering systems, video captioning and activity recognition, large-scale human detection and tracking in videos, or anomaly detection in images for factory automation.

#### **Author details**

Cristiano Premebida<sup>1</sup> \*, Rares Ambrus<sup>2</sup> and Zoltan-Csaba Marton<sup>3</sup>


#### **References**

[1] Russell SJ, Norvig P. Artificial Intelligence: A Modern Approach. 2nd ed. Upper Saddle River, New Jersey: Prentice Hall, ISBN 0-13-790395-2; 2003

[2] Rennie C, Shome R, Bekris KE, Souza AF. A dataset for improved RGBD-based object detection and pose estimation for warehouse pick-and-place. IEEE Robotics and Automation Letters. July 2016;**1**(2), pp. 1179-1185

[17] Sünderhauf N, Brock O, Scheirer W, Hadsell R, Fox D, Leitner J, Upcroft B, Abbeel P, Burgard W, Milford M, Corke P. The limits and potentials of deep learning for robotics.

Intelligent Robotic Perception Systems http://dx.doi.org/10.5772/intechopen.79742 125

[18] Correll N, Bekris KE, Berenson D, Brock O, Causo A, Hauser K, Okada K, Rodriguez A, Romano JM, Wurman PR, et al. IEEE Transactions on Automation Science and Engineering.

[19] Wagstaff K. Machine learning that matters. In: Proceedings of the Twenty-Ninth

[20] Moravec H, Elfes A. High resolution maps from wide angle sonar. In: Proceedings of IEEE International Conference on Robotics and Automation; 1985. pp. 116-121

[21] Hornung A, Wurm KM, Bennewitz M, Stachniss C, Burgard W. OctoMap: An efficient probabilistic 3D mapping framework based on octrees. Autonomous Robots. 2013 [22] Thrun S, Burgard W, Fox D. Probabilistic Robotics. The MIT Press; 2005. ISBN:0262201623 [23] Cadena C, Carlone L, Carrillo H, Latif Y, Scaramuzza D, Neira J, Reid I, Leonard JJ. Past, present, and future of simultaneous localization and mapping: Toward the robust-

[24] Biber P, Ducket T. Experimental analysis of sample-based maps for long-term SLAM. The

[25] Walcott-Bryant A, Kaess M, Johannsson H, Leonard JJ. Dynamic pose graph SLAM: Longterm mapping in low dynamic environments. In: IEEE/RSJ International Conference on

[26] Saarinen J, Stoyanov T, Andreasson H, Lilienthal AJ. Fast 3D mapping in highly dynamic environments using normal distributions transform occupancy maps. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); 2013. pp. 4694-4701

[27] Hermans A, Floros G, Leibe B. Dense 3D semantic mapping of indoor scenes from RGB-D images. In: IEEE International Conference on Robotics and Automation (ICRA), Hong

[28] Krahenbuhl P, Koltun V. Efficient inference in fully connected CRFs with Gaussian edge potentials. Advances in Neural Information Processing Systems. 2016;**24**:109-117 [29] McCormac J, Handa A, Davison A, Leutenegger S. SemanticFusion: Dense 3D semantic mapping with convolutional neural networks. In: IEEE International Conference on

[30] Whelan T, Leutenegger S, Salas-Moreno RF, Glocker B, Davison AJ. ElasticFusion: Dense

[31] Ambrus R, Claici S, Wendt A. Automatic room segmentation from unstructured 3-D data of indoor environments. IEEE Robotics and Automation Letters. 2017;**2**(2):749-756

[32] Ambrus R, Bore N, Folkesson J, Jensfelt P. Autonomous meshing, texturing and recognition of object models with a mobile robot. In: IEEE/RSJ International Conference on

Intelligent Robots and Systems (IROS), Vancouver, BC. 2017. pp. 5071-5078

International Conference on Machine Learning (ICML); 2012. pp. 529-536

perception age. IEEE Transactions on Robotics. 2016;**32**(6):1309-1332

International Journal of Robotics Research. 2009;**28**:20-33

Intelligent Robots and Systems (IROS); 2012. pp. 1871-1878

Robotics and Automation (ICRA), Singapore; 2017. pp. 4628-4635

SLAM without a pose graph. Robotics: Science and Systems. 2015

Kong; 2014. pp. 2631-2638

The International Journal of Robotics Research. 2018;**37**(4-5):405-420

2016


[17] Sünderhauf N, Brock O, Scheirer W, Hadsell R, Fox D, Leitner J, Upcroft B, Abbeel P, Burgard W, Milford M, Corke P. The limits and potentials of deep learning for robotics. The International Journal of Robotics Research. 2018;**37**(4-5):405-420

[2] Rennie C, Shome R, Bekris KE, Souza AF. A dataset for improved RGBD-based object detection and pose estimation for warehouse pick-and-place. IEEE Robotics and Auto-

[3] Bore N, Jensfelt P, Folkesson J. Multiple object detection, tracking and long-term dynam-

[4] Handa A, Patraucean V, Badrinarayanan V, Stent S, Cipolla R. Understanding real world indoor scenes with synthetic data. 2016 IEEE Conference on Computer Vision and

[5] Firman M. RGBD datasets: Past, present and future. Firman 2016 RGBDDP, In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) on

[6] Sünderhauf N, Dayoub F, McMahon S, Talbot B, Schulz R, Corke P, Wyeth G, Upcroft B, Milford M. Place categorization and semantic mapping on a mobile robot. In: IEEE International Conference on Robotics and Automation (ICRA); Stockholm; 2016. pp. 5729-5736 [7] Brucker M, Durner M, Ambrus R, Csaba Marton Z, Wendt A, Jensfelt P, Arras KO, Triebel R. Semantic labeling of indoor environments from 3D RGB maps. In: IEEE/RSJ

[8] Saarinen J, Andreasson H, Lilienthal AJ. Independent Markov chain occupancy grid maps for representation of dynamic environment. In: IEEE/RSJ International Conference

[9] Fong T, Nourbakhsh I, Dautenhahn K. A survey of socially interactive robots. Robotics

[10] Diego R. Faria, Mario Vieira, Premebida C, Nunes U. Probabilistic human daily activity recognition towards robot-assisted living. In: Proceedings of the IEEE RO-MAN'15;

[11] Manduchi R, Castano A, Talukder A, Matthies L. Obstacle detection and terrain classification for autonomous off-road navigation. Autonomous Robots. 2005;**18**(1):81-102 [12] Fernandes R, Premebida C, Peixoto P, Wolf D, Nunes U. Road detection using high resolution LIDAR. In: IEEE Vehicle Power and Propulsion Conference, IEEE-VPPC; 2014

[13] Asvadi A, Garrote L, Premebida C, Peixoto P, Nunes UJ. Multimodal vehicle detection: Fusing 3D LIDAR and color camera data. Pattern Recognition Letters. Elsevier. 2017 [14] Premebida C, Nunes U. Fusing LIDAR, camera and semantic information: A contextbased approach for pedestrian detection. The International Journal of Robotics Research.

[15] Dondrup C, Bellotto N, Jovan F, Hanheide M. Real-time multisensor people tracking for human-robot spatial interaction. IEEE International Conference on Robotics and Automation

[16] Andreasson H, Magnusson M, Lilienthal A. Has something changed here? Autonomous difference detection for security patrol robots. In: IEEE/RSJ International Conference on

Intelligent Robots and Systems (IROS); 2007. pp. 3429-3435

ics learning in large 3D maps. CoRR, https://arxiv.org/abs/1801.09292. 2018

Large Scale 3D Data: Acquisition, Modelling and Analysis; 2016. 661-673

International Conference on Intelligent Robots and Systems (IROS); 2018

on Intelligent Robots and Systems (IROS); 2012. pp. 3489-3495

and Autonomous Systems. 2003;**42**(3-4):143-166

Japan; 2015

124 Applications of Mobile Robots

2013;**32**(3):371-384

(ICRA); 2015

Pattern Recognition (CVPR), Las Vegas, NV, 2016, pp. 4077-4085

mation Letters. July 2016;**1**(2), pp. 1179-1185


[33] Armeni I, Sener O, Zamir AR, Jiang H, Brilakis I, Fischer M, Savarese S. 3D semantic parsing of large-scale indoor spaces. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV; 2016. pp. 1534-1543

[49] Lepetit V, Moreno-Noguer F, Fua P. Epnp: An accurate o(n) solution to the pnp problem.

Intelligent Robotic Perception Systems http://dx.doi.org/10.5772/intechopen.79742 127

[50] Janai J, Guney F, Behl A, Geiger A. Computer vision for autonomous vehicles: Problems,

[51] Tas OS, Salscheider NO, Poggenhans F, Wirges S, Bandera C, Zofka MR, Strauss T, Zollner JM, Stiller C. Making Bertha Cooperate—Team AnnieWAY's entry to the 2016 Grand Cooperative Driving Challenge. IEEE Transactions on Intelligent Transportation Systems.

[52] Hawes N, Burbridge C, Jovan F, Kunze L, Lacerda B, Mudrova L, Young J, Wyatt J, Hebesberger D, Kortner T, Ambrus R, Bore N, Folkesson J, Jensfelt P, Beyer L, Hermans A, Leibe B, Aldoma A, Faulhammer T, Zillich M, Vincze M, Chinellato E, Al-Omari M, Duckworth P, Gatsoulis Y, Hogg DC, Cohn AG, Dondrup C, Pulido Fentanes J, Krajnik T, Santos JM, Duckett T, Hanheide M. The STRANDS project: Long-term autonomy in every-

[53] Fäulhammer T, Ambruş R, Burbridge C, Zillich M, Folkesson J, Hawes N, Jensfelt P, Vincze M. Autonomous learning of object models on a mobile robot. IEEE Robotics and

[54] Krajnik T, Fentanes JP, Cielniak G, Dondrup C, Duckett T. Spectral analysis for longterm robotic mapping. In: IEEE International Conference on Robotics and Automation

[55] Beyer L, Hermans A, Leibe B. Biternion nets: Continuous head pose regression from discrete training labels. German Conference on Pattern Recognition. 2014:157-168 [56] Beyer L, Hermans A, Leibe B. DROW: Real-time deep learning-based wheelchair detection in 2-D range data. IEEE Robotics and Automation Letters. 2017;**2**(2):585-592

[57] Fentanes JP, Lacerda B, Krajnik T, Hawes N, Hanheide M. Now or later? Predicting and maximising success of navigation actions from long-term experience. In: IEEE International Conference on Robotics and Automation (ICRA), Seattle, USA. 2015.

[58] Santos JM, Krajník T, Fentanes JP, Duckett T. Lifelong information-driven exploration to complete and refine 4-D spatio-temporal maps. IEEE Robotics and Automation Letters.

[59] Triebel R, Arras K, Alami R, Beyer L, Breuers S, Chatila R, Chetouani M, Cremers D, Evers V, Fiore M, Hung H, Islas Ramírez OA, Joosse M, Khambhaita H, Kucner T, Leibe B, Lilienthal AJ, Linder T, Lohse M, Magnusson M, Okal B, Palmieri L, Rafi U, van Rooij M, Zhang L. SPENCER: A socially aware service robot for passenger guidance and help in busy airports. Field and Service Robotics: Results of the 10th International Conference.

[60] Szeliski R. Computer vision: Algorithms and applications ser. In: Texts in Computer

Science. New York, NY, USA: Springer-Verlag New York, Inc.; 2010

day environments. IEEE Robotics and Automation Magazine. 2017;**24**:146-156

International Journal of Computer Vision. 2009;**81**(2)

datasets and state-of-the-art. arXiv:1704.05519v1. 2018

2018;**19**(4):1262-1276

pp. 1112-1117

2016;**1**(2):684-691

2016. 607-622

Automation Letters. 2017;**2**(1):26-33

(ICRA), Hong Kong. 2014. pp. 3706-3711


[49] Lepetit V, Moreno-Noguer F, Fua P. Epnp: An accurate o(n) solution to the pnp problem. International Journal of Computer Vision. 2009;**81**(2)

[33] Armeni I, Sener O, Zamir AR, Jiang H, Brilakis I, Fischer M, Savarese S. 3D semantic parsing of large-scale indoor spaces. In: IEEE Conference on Computer Vision and Pattern

[34] Mura C, Mattausch O, Pajarola R. Piecewise-planar reconstruction of multi-room interiors with arbitrary wall arrangements. Computer Graphics Forum. 2016;**35**:179-188 [35] Sperry RW. Neurology and the mind-body problem. American Scientist. 1952;**40**:291-312 [36] Turing AM. Intelligent machinery. Journal of Machine Intelligence. 1970;**5**, different

[38] Steels L, Brooks RA, editors. The Artificial Life Route to Artificial Intelligence: Building Embodied, Situated Agents. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Inc.; 1995

[39] Vernon D. Cognitive vision: The case for embodied perception. In: Image and Vision

[40] Bohg J, Hausman K, Sankaran B, Brock O, Kragic D, Schaal S, Sukhatme G. Interactive perception: Leveraging action in perception and perception in action. IEEE Transactions

[41] Ferrari C, Canny J. Planning optimal grasps. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA); Vol. 3; 1992. pp. 2290-2295

[42] Frisby JP, Stone JV. Seeing objects, ch. 8. In: Seeing: The Computational Approach to

[43] Ulrich M, Wiedemann C, Steger C. Cad-based recognition of 3d objects in monocular images. In: IEEE International Conference on Robotics and Automation (ICRA); Vol. 9;

[44] Hinterstoisser S, Lepetit V, Ilic S, Holzer S, Bradski G, Konolige K, Navab N. Model based training, detection and pose estimation of texture-less 3d objects in heavily cluttered

[45] Tjaden H, Schwanecke U, Schomer E. Real-time monocular pose estimation of 3d objects using temporally consistent local color histograms. In: Computer Vision (ICCV), 2017

[46] Brachmann E, Krull A, Michel F, Gumhold S, Shotton J, Rother C. Learning 6d object pose estimation using 3d object coordinates. In: European Conference on Computer Vision.

[47] Krull A, Michel F, Brachmann E, Gumhold S, Ihrke S, Rother C. 6-DOF model based tracking via object coordinate regression. In: Asian Conference on Computer Vision. Springer;

[48] Kehl W, Milletari F, Tombari F, Ilic S, Navab N. Deep learning of local rgb-d patches for 3d object detection and 6d pose estimation. In: European Conference on Computer Vision.

scenes. In: Asian conference on computer vision; Springer; 2012. pp. 548-562

IEEE International Conference on. IEEE; 2017. pp. 124-132

[37] Turing AM. Computing machinery and intelligence. Mind. 1950;**LIX**:433-460

Recognition (CVPR), Las Vegas, NV; 2016. pp. 1534-1543

sources cite 1947 and 1948 as the time of writing

Computing. Elsevier. 1 January 2008;**26**(1):127-140

Biological Vision. MIT Press; 2010. pp. 178-179

on Robotics. 2017;**33**:1273-1291

2009. pp. 1191-1198

126 Applications of Mobile Robots

Springer; 2014. pp. 536-551

Springer; 2016. pp. 205-220

2014. pp. 384-399


**Chapter 7**

Provisional chapter

**Online Mapping-Based Navigation System for Wheeled**

DOI: 10.5772/intechopen.79412

A road mapping and feature extraction for mobile robot navigation in road roundabout and road following environments is presented in this chapter. In this work, the online mapping of mobile robot employing the utilization of sensor fusion technique is used to extract the road characteristics that will be used with path planning algorithm to enable the robot to move from a certain start position to predetermined goal, such as road curbs, road borders, and roundabout. The sensor fusion is performed using many sensors, namely, laser range finder, camera, and odometry, which are combined on a new wheeled mobile robot prototype to determine the best optimum path of the robot and localize it within its environments. The local maps are developed using an image's preprocessing and processing algorithms and an artificial threshold of LRF signal processing to recognize the road environment parameters such as road curbs, width, and roundabout. The path planning in the road environments is accomplished using a novel approach so called Laser Simulator to find the trajectory in the local maps developed by sensor fusion. Results show the capability of the wheeled mobile robot to effectively recognize the road environments, build a local mapping, and find the path in both road following and

Keywords: LRF, vision system, odometry, laser simulator, road roundabout,

Autonomous navigation can be defined as the ability of the mobile robot to determine its position within the reference frame environment using suitable sensors, plan its path mission through the terrain from the start toward the goal position using high planner techniques and

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

Online Mapping-Based Navigation System for Wheeled

**Mobile Robot in Road Following and Roundabout**

Mobile Robot in Road Following and Roundabout

Mohammed A. H. Ali and Musa Mailah

Mohammed A. H. Ali and Musa Mailah

http://dx.doi.org/10.5772/intechopen.79412

Abstract

roundabout.

1. Introduction

road following, 2D map, local map

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

#### **Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout** Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

DOI: 10.5772/intechopen.79412

Mohammed A. H. Ali and Musa Mailah Mohammed A. H. Ali and Musa Mailah

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79412

#### Abstract

A road mapping and feature extraction for mobile robot navigation in road roundabout and road following environments is presented in this chapter. In this work, the online mapping of mobile robot employing the utilization of sensor fusion technique is used to extract the road characteristics that will be used with path planning algorithm to enable the robot to move from a certain start position to predetermined goal, such as road curbs, road borders, and roundabout. The sensor fusion is performed using many sensors, namely, laser range finder, camera, and odometry, which are combined on a new wheeled mobile robot prototype to determine the best optimum path of the robot and localize it within its environments. The local maps are developed using an image's preprocessing and processing algorithms and an artificial threshold of LRF signal processing to recognize the road environment parameters such as road curbs, width, and roundabout. The path planning in the road environments is accomplished using a novel approach so called Laser Simulator to find the trajectory in the local maps developed by sensor fusion. Results show the capability of the wheeled mobile robot to effectively recognize the road environments, build a local mapping, and find the path in both road following and roundabout.

Keywords: LRF, vision system, odometry, laser simulator, road roundabout, road following, 2D map, local map

#### 1. Introduction

Autonomous navigation can be defined as the ability of the mobile robot to determine its position within the reference frame environment using suitable sensors, plan its path mission through the terrain from the start toward the goal position using high planner techniques and

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

perform the path using actuators, all with a high degree of autonomy. In other words, the robot during navigation must be able to answer the following questions [1]:

orientation or position error. The kinematic and dynamic analysis of the mobile robot is used to find the parameters of the controllers for continuously avoiding the obstacles and moving toward the target position through the predefined path [1]. The autonomous navigation

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

http://dx.doi.org/10.5772/intechopen.79412

131

i. Structured environments: it is usually found in indoor applications, where the environ-

iii. Unstructured environments: outdoor applications are almost unstructured, where the

There are many factors that can make outdoor autonomous navigation very complex in

i. The borders of surrounding terrain in the indoor application are clear and distinct.

ii. The robot algorithm should be able to discover and deal with many things that are not in

iv. Outdoor autonomous navigation should have adequate robustness and reliable enough because the robot typically works and meddles with people and encounters unexpected

v. Weather changes can affect the mobile robot sensor measurement, electronic devices, and actuators. For example, the inhomogeneous of the light will affect the camera when capturing the image of the scene, the sunray affects the laser measurement, and ultrasonic

Although the outdoor navigation in urban area is a topic involving a wide range and level of difficulties, it is attractive and forms a challenge to the researchers who are motivated to discover solution(s) to the problems via their proposed developed technique. Until today, there is as yet a robot or vehicle, which is able to drive fully autonomously on the roads of urban buildings, taking into account a high level of robust interaction with its environments [3, 4]. This vehicle is expected to operate in a dangerous situation, since it always passes through crowded area, must follow printed or signal traffic lights and more often than not, it is in touch

Most navigation systems on the roads are performed using two or more sensors to obtain the parameters of the surrounding environment, taking into consideration various situations and conditions. Sensor fusion technique is the common method that has been used in navigation of mobile robot in road environments. In this model, the methods for extracting the features of roads can be done with reference to either behavior-based navigation or model-based naviga-

Global positioning system (GPS) is used as the main sensor and combined with odometry and LRF for trajectory tracking, obstacle avoiding and localization in curbed roads [5] or navigating

However, in contrary to that, they are mostly "faded" in outdoor setting.

iii. The locomotion of robot will be different depending on the terrain roughness.

system can be classified according to the environmental features as follows:

ii. Semi-structured environments: the environments are partially known.

ments are totally known.

comparison to the indoor navigation:

the initial planning.

moving obstacles.

with the people.

environments are totally unknown.

measurement becomes unavailable.

tion as described in the following paragraphs.


Autonomous navigation comprises many tasks between the sensing and actuating processes that can be further divided into two approaches: behavior-based navigation and model-based navigation. The behavior-based navigation depends on the interaction of some asynchronous sets of behaviors of the robot and the environment such as: build maps, explore, avoid obstacles, and follow right/left and goal seeking, which are fused together to generate suitable actions for the actuators. The model-based navigation includes four subtasks that are performed synchronously step by step: perception, localization, path planning, and motion control. This is the common navigation systems that have been used and found in the literature.

The model-based navigation will be explained in details, since it is implemented in the proposed navigation system. In the perception task of this navigation model, the sensors are used to acquire and produce enormous information and good observation for the environments, where the robot navigates. There is a range of sensors that can be chosen for the autonomous navigation depending on the task of mobile robot, which generally can be classified into two groups: absolute position measurements and relative position measurements [2]. The European robotics platform (EUROP) marks the perception process in the robotics system as the main problem which needs further research solutions [1].

The localization process can be perceived as the answer to the question: where am I? In order to enable the robot to find its location within the environments, two types of information are needed. First, a-priori information is given to the robot from maps or cause-effect relationships in an initialization phase. Second, the robot acquires information about the environment through observation by sensors. Sensor fusion techniques are always used to combine the initial information and position measurements of sensors to estimate the location of the robot instantly [1].

In the path planning process, the collision-free path between start and target positions is determined continuously. There are two types of path planning: (a) global planning or deliberative technique and (b) local path planning or sensor-based planning. In the former, the surrounding terrain of the mobile robot is known totally, and then the collision-free path is determined offline, while in the latter, the surrounding terrain of the mobile robot is partially or totally unknown, and the robot uses the sensor data for planning the path online in the environments [1]. The path planning consists of three main steps, namely, the environment's modeling, determination of the collision-free trajectory from the initial to target positions, and searching continuously for the goal [1].

In motion control, the mobile robot should apply a steering strategy that attempts to prevent slippage and reduce position errors. The control algorithm has to guarantee a zero steady-state orientation or position error. The kinematic and dynamic analysis of the mobile robot is used to find the parameters of the controllers for continuously avoiding the obstacles and moving toward the target position through the predefined path [1]. The autonomous navigation system can be classified according to the environmental features as follows:


perform the path using actuators, all with a high degree of autonomy. In other words, the robot

Autonomous navigation comprises many tasks between the sensing and actuating processes that can be further divided into two approaches: behavior-based navigation and model-based navigation. The behavior-based navigation depends on the interaction of some asynchronous sets of behaviors of the robot and the environment such as: build maps, explore, avoid obstacles, and follow right/left and goal seeking, which are fused together to generate suitable actions for the actuators. The model-based navigation includes four subtasks that are performed synchronously step by step: perception, localization, path planning, and motion control. This is the common

The model-based navigation will be explained in details, since it is implemented in the proposed navigation system. In the perception task of this navigation model, the sensors are used to acquire and produce enormous information and good observation for the environments, where the robot navigates. There is a range of sensors that can be chosen for the autonomous navigation depending on the task of mobile robot, which generally can be classified into two groups: absolute position measurements and relative position measurements [2]. The European robotics platform (EUROP) marks the perception process in the robotics system as the main problem

The localization process can be perceived as the answer to the question: where am I? In order to enable the robot to find its location within the environments, two types of information are needed. First, a-priori information is given to the robot from maps or cause-effect relationships in an initialization phase. Second, the robot acquires information about the environment through observation by sensors. Sensor fusion techniques are always used to combine the initial information and position measurements of sensors to estimate the location of the robot

In the path planning process, the collision-free path between start and target positions is determined continuously. There are two types of path planning: (a) global planning or deliberative technique and (b) local path planning or sensor-based planning. In the former, the surrounding terrain of the mobile robot is known totally, and then the collision-free path is determined offline, while in the latter, the surrounding terrain of the mobile robot is partially or totally unknown, and the robot uses the sensor data for planning the path online in the environments [1]. The path planning consists of three main steps, namely, the environment's modeling, determination of the collision-free trajectory from the initial to target positions, and

In motion control, the mobile robot should apply a steering strategy that attempts to prevent slippage and reduce position errors. The control algorithm has to guarantee a zero steady-state

during navigation must be able to answer the following questions [1]:

• Where have I been? It is solved using cognitive maps.

• Where am I going? It is done by path planning.

130 Applications of Mobile Robots

which needs further research solutions [1].

searching continuously for the goal [1].

instantly [1].

• Where am I? It is determined by the localization algorithm.

• How can I go there? It is performed by motion control system.

navigation systems that have been used and found in the literature.

iii. Unstructured environments: outdoor applications are almost unstructured, where the environments are totally unknown.

There are many factors that can make outdoor autonomous navigation very complex in comparison to the indoor navigation:


Although the outdoor navigation in urban area is a topic involving a wide range and level of difficulties, it is attractive and forms a challenge to the researchers who are motivated to discover solution(s) to the problems via their proposed developed technique. Until today, there is as yet a robot or vehicle, which is able to drive fully autonomously on the roads of urban buildings, taking into account a high level of robust interaction with its environments [3, 4]. This vehicle is expected to operate in a dangerous situation, since it always passes through crowded area, must follow printed or signal traffic lights and more often than not, it is in touch with the people.

Most navigation systems on the roads are performed using two or more sensors to obtain the parameters of the surrounding environment, taking into consideration various situations and conditions. Sensor fusion technique is the common method that has been used in navigation of mobile robot in road environments. In this model, the methods for extracting the features of roads can be done with reference to either behavior-based navigation or model-based navigation as described in the following paragraphs.

Global positioning system (GPS) is used as the main sensor and combined with odometry and LRF for trajectory tracking, obstacle avoiding and localization in curbed roads [5] or navigating mobile robot between urban buildings [6]. GPS can be combined with LRF and dead reckoning for navigating vehicles in roads using beacons and landmarks [7], combined with camera vision for localizing mobile robot in color marked roads [8], combined with 3D laser scanner for building compact maps to be used in a miscellaneous navigation applications of mobile robot [9], combined with IMU, camera vision, and sonar altimeter for navigating unmanned helicopter through urban building [10] or combined with LRF and inertial sensors for leading mobile robot ATRV-JR in paved and rugged terrain [11, 12]. GPS can be combined with INS and odometry sensor for helping GPS to increase its positioning accuracy of vehicles [13]. GPS is combined with odometry and GIS (geographical information system) for road matching-based vehicle navigation [14]. GPS can also be combined with LIDAR (light detection and ranging) and INS for leading wheelchair in urban building by 3D map-based navigation [15]. GPS is combined with a video camera and INS for navigating the vehicles by lane detection of roads [16]. GPS is combined with INS for increasing position estimation of vehicles [17]. GPS is combined with INS and odometry for localizing mobile robot in urban buildings [18]. Camera video with odometry is used for navigating land vehicle, road following by lane signs and obstacles avoiding [19]. Omni-directional infrared vision system can be used for localizing patrol mobile robot in an electrical station environment [20]. 3D map building and urban scenes are used in mobile robot navigation by fusing stereo vision and LRF [21].

using LRF in a road environment. The information filter (derived from Kalman filter) is used to gather data from sensors and build map and determine the localization of robot. The results show that the accuracy of the online mapping is very close to DGPS, which has accuracy equal to 1 cm. Donghoon et al. [8] proposed a navigation system method based on fusing two localization systems is proposed. The first one uses a camera vision system with particle filter, while the second uses two GPS with Kalman filter. The printed marks on the road like arrows, lines, and cross are effectively detected through the camera system. By processing the camera's data using the hyperbola-pair lane detection, that is, Delaunay triangulation method for point extraction and mapping based on sky view of points, the robot is able to determine the current position within its environment. To eliminate the camera drawback like lens distortion and inaccuracy in the long run, GPS system data are used to complement and validate the camera's data. The errors between the predefined map and real-time measurements are found to be

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

http://dx.doi.org/10.5772/intechopen.79412

133

Cristina et al. [9] suggested a compact 3D map building for a miscellaneous navigation application of mobile robot in real time performed through the use of the GPS and 3D laser scanner. A 3D laser scanner senses the environment surface with sufficient accuracy to obtain a map, which is processed to get a qualitative description of the traversable environment cell. The 3D local map is built using 2D Voronoi diagram model, while the third dimension is determined through the data from laser scanner for the terrain reversibility. The previous map is gathered with the GPS information to build a global map. This system is able to detect the slopes of navigated surfaces using Brezets algorithm and the roughness of the terrain using normal vector deviation for the whole region. Although the proposed algorithm is very complicated, the result shows that it is

A new platform for an autonomous wheeled mobile robot navigation system has been designed and developed in our labs. It is comprised of the following parts as shown in

• Five cards for the interface free-based controllers system (IFC): interface power, abbreviated as IFC-IP00; interface computer, abbreviated as IFC-IC00; interface brushless, abbre-

viated as IFC-BL02; and interface brush motors, abbreviated as IFCBH02.

able to extract an area as big as 40 20 m in an outdoor environment for 0.89 s.

• Two motors, type: DC-brush with power 120 W and model DKM.

• 4 m range LRF; model HOKUYO URG-04LX-UG01. • High resolution WiFi camera, model JVC-GC-XA1B.

• Two optical rotary odometry; model, B106.

• Motors drivers, types SmartDrive 40.

0.78 m in x direction and 0.43 m in y direction.

2. Platform overview

• One spherical Castor wheel.

Figure 1:

LRF can be combined with cameras and odometry for online modeling of road boundary navigation [6], for enhancing position accuracy during mobile robot navigation [22] or for correcting trajectory navigation of mobile robot [23]. Also, it can be combined with compass and odometry for building map-based mobile robot navigation [1]. It can be combined with color CCD camera for modeling roads and driving mobile robot in paved and unpaved road [24], for crossing roads through landmark detection [25], for obstacle detecting and avoiding [26] or for road recognition process [27]. It can be combined with sonar and camera vision for navigating mobile robot when crossing roads [1]. 3D LRF is combined with two color camera, INS, and odometry for cross-country navigation of ADAM mobile robot through natural terrain [1]. It can be combined with IMU, CCD camera, and odometry for guiding tractor vehicle in Citrus Grove [1]. It can be combined with camera, LRF, sonar, and compass for building 3D map of an urban environment [1].

Priori map can be used for navigation by mapping precisely environment landmarks [1]. Dead reckoning is used for estimating position accurately by feature detection in mobile robot [1]. A hybrid methodology between the teleoperating and autonomous navigation system has been applied in a curbed road environment as explained in [5]. A combination of differential global positioning system (DGPS) and odometry with extended Kalman filter is used to localize the mobile robot in a road environment. LRF is used for obstacle avoidance by finding the suitable path to pass through the road curbs and to estimate the position of the road curbs during trajectory tracking. The main setback in this work is that the robot cannot work autonomously in road environments.

Jose et al. [7] presented a robust outdoor navigation system using LRF, and dead reckoning is proposed for navigating vehicles using beacons and landmarks on the roads. The system is studied considering several cylindrical or V-shaped objects (beacons) that can be detected using LRF in a road environment. The information filter (derived from Kalman filter) is used to gather data from sensors and build map and determine the localization of robot. The results show that the accuracy of the online mapping is very close to DGPS, which has accuracy equal to 1 cm. Donghoon et al. [8] proposed a navigation system method based on fusing two localization systems is proposed. The first one uses a camera vision system with particle filter, while the second uses two GPS with Kalman filter. The printed marks on the road like arrows, lines, and cross are effectively detected through the camera system. By processing the camera's data using the hyperbola-pair lane detection, that is, Delaunay triangulation method for point extraction and mapping based on sky view of points, the robot is able to determine the current position within its environment. To eliminate the camera drawback like lens distortion and inaccuracy in the long run, GPS system data are used to complement and validate the camera's data. The errors between the predefined map and real-time measurements are found to be 0.78 m in x direction and 0.43 m in y direction.

Cristina et al. [9] suggested a compact 3D map building for a miscellaneous navigation application of mobile robot in real time performed through the use of the GPS and 3D laser scanner. A 3D laser scanner senses the environment surface with sufficient accuracy to obtain a map, which is processed to get a qualitative description of the traversable environment cell. The 3D local map is built using 2D Voronoi diagram model, while the third dimension is determined through the data from laser scanner for the terrain reversibility. The previous map is gathered with the GPS information to build a global map. This system is able to detect the slopes of navigated surfaces using Brezets algorithm and the roughness of the terrain using normal vector deviation for the whole region. Although the proposed algorithm is very complicated, the result shows that it is able to extract an area as big as 40 20 m in an outdoor environment for 0.89 s.

#### 2. Platform overview

mobile robot between urban buildings [6]. GPS can be combined with LRF and dead reckoning for navigating vehicles in roads using beacons and landmarks [7], combined with camera vision for localizing mobile robot in color marked roads [8], combined with 3D laser scanner for building compact maps to be used in a miscellaneous navigation applications of mobile robot [9], combined with IMU, camera vision, and sonar altimeter for navigating unmanned helicopter through urban building [10] or combined with LRF and inertial sensors for leading mobile robot ATRV-JR in paved and rugged terrain [11, 12]. GPS can be combined with INS and odometry sensor for helping GPS to increase its positioning accuracy of vehicles [13]. GPS is combined with odometry and GIS (geographical information system) for road matching-based vehicle navigation [14]. GPS can also be combined with LIDAR (light detection and ranging) and INS for leading wheelchair in urban building by 3D map-based navigation [15]. GPS is combined with a video camera and INS for navigating the vehicles by lane detection of roads [16]. GPS is combined with INS for increasing position estimation of vehicles [17]. GPS is combined with INS and odometry for localizing mobile robot in urban buildings [18]. Camera video with odometry is used for navigating land vehicle, road following by lane signs and obstacles avoiding [19]. Omni-directional infrared vision system can be used for localizing patrol mobile robot in an electrical station environment [20]. 3D map building and urban scenes are used in

LRF can be combined with cameras and odometry for online modeling of road boundary navigation [6], for enhancing position accuracy during mobile robot navigation [22] or for correcting trajectory navigation of mobile robot [23]. Also, it can be combined with compass and odometry for building map-based mobile robot navigation [1]. It can be combined with color CCD camera for modeling roads and driving mobile robot in paved and unpaved road [24], for crossing roads through landmark detection [25], for obstacle detecting and avoiding [26] or for road recognition process [27]. It can be combined with sonar and camera vision for navigating mobile robot when crossing roads [1]. 3D LRF is combined with two color camera, INS, and odometry for cross-country navigation of ADAM mobile robot through natural terrain [1]. It can be combined with IMU, CCD camera, and odometry for guiding tractor vehicle in Citrus Grove [1]. It can be combined with camera, LRF, sonar, and compass for

Priori map can be used for navigation by mapping precisely environment landmarks [1]. Dead reckoning is used for estimating position accurately by feature detection in mobile robot [1]. A hybrid methodology between the teleoperating and autonomous navigation system has been applied in a curbed road environment as explained in [5]. A combination of differential global positioning system (DGPS) and odometry with extended Kalman filter is used to localize the mobile robot in a road environment. LRF is used for obstacle avoidance by finding the suitable path to pass through the road curbs and to estimate the position of the road curbs during trajectory tracking. The main setback in this work is that the robot cannot work autonomously

Jose et al. [7] presented a robust outdoor navigation system using LRF, and dead reckoning is proposed for navigating vehicles using beacons and landmarks on the roads. The system is studied considering several cylindrical or V-shaped objects (beacons) that can be detected

mobile robot navigation by fusing stereo vision and LRF [21].

building 3D map of an urban environment [1].

in road environments.

132 Applications of Mobile Robots

A new platform for an autonomous wheeled mobile robot navigation system has been designed and developed in our labs. It is comprised of the following parts as shown in Figure 1:


The first and second steps will be explained here, while the third one will be explained in next

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

http://dx.doi.org/10.5772/intechopen.79412

135

A preprocessing algorithm is utilized to capture the environments of the road from the live video and prepare it in an appropriate form to be processed in the next processes. The main

• Constructing the video input object: the video input object represents the connection

• Preview of the WiFi video: it creates a video preview window that displays live video data for video input object. The window also displays the timestamp and video resolution of

The brightness of image's sequences is adjusted, since the aperture of camera is not

This function helps to reserve the image device for the exclusive use in this program and locks the configuration to others applications. Thus, certain properties become read only

It returns the data, which contains the number of frames specified in the (Frames per

It allows for cutting the interesting regions in the images. In the proposed algorithm, the bottom half of the image is cropped, since it contains the nearest area of the road to the

The purpose is to make image useful for the edge detection and removing the noise operations.

robot; also to save time needed for calculation in comparison with whole image.

operations that have been used at this step can be briefly described as follows:

between MATLAB and a particular image acquisition device.

each frame and the current status of video input object.

(i) Preprocessing of the image for depth processing.

vid = videoinput('winvideo',3);

• Setting the brightness of live video

set (vid.source, 'Brightness', 35);

automatic. Following command is used:

• Acquiring the image frames to MATLAB workspace

Trigger) property of the video input object.

GH = imcrop(data, [1 u1 io(2) io(1)]);

• Convert from RGB into grayscale

preview (vid);

• Start acquiring frames

while running.

data = getdata(vid);

IS = rgb2gray(GH);

Start(vid);

• Crop image

section.

Figure 1. The developed platform used for experiments.

• The platform has an additional part such as battery with model NP7-12 lead acid and the aluminum profile and sheet-based chassis as shown in Figure 1.

#### 3. Sensor modeling and feature extraction

The data coming from each sensor in the platform were prepared in such a way that it enables for the extraction of the road features autonomously while navigating on the road. It is described in the following section.

#### 3.1. Camera

JVC-GC-XA1B camera is utilized to figure out the environments in front of the robot with good resolution (760 320) and speed equal to 30 frame/s. The sequences of pictures are extracted from the live video using image processing tool boxes in MATLAB. An algorithm has been developed that can take the image sequences from the video and apply multiple operations to get a local map from the image and perform further calculation for road following and roundabout detection. In general, the image sequences processing algorithm consists of three main parts:

	- Roundabout detection based on LS approach.
	- Road following in the roads without curbs.

The first and second steps will be explained here, while the third one will be explained in next section.

(i) Preprocessing of the image for depth processing.

A preprocessing algorithm is utilized to capture the environments of the road from the live video and prepare it in an appropriate form to be processed in the next processes. The main operations that have been used at this step can be briefly described as follows:

• Constructing the video input object: the video input object represents the connection between MATLAB and a particular image acquisition device.

```
vid = videoinput('winvideo',3);
```
• Preview of the WiFi video: it creates a video preview window that displays live video data for video input object. The window also displays the timestamp and video resolution of each frame and the current status of video input object.

preview (vid);

• The platform has an additional part such as battery with model NP7-12 lead acid and the

The data coming from each sensor in the platform were prepared in such a way that it enables for the extraction of the road features autonomously while navigating on the road. It is

JVC-GC-XA1B camera is utilized to figure out the environments in front of the robot with good resolution (760 320) and speed equal to 30 frame/s. The sequences of pictures are extracted from the live video using image processing tool boxes in MATLAB. An algorithm has been developed that can take the image sequences from the video and apply multiple operations to get a local map from the image and perform further calculation for road following and roundabout detection. In general, the image sequences processing algorithm consists of three

aluminum profile and sheet-based chassis as shown in Figure 1.

3. Sensor modeling and feature extraction

Figure 1. The developed platform used for experiments.

i. Preprocessing of the image for depth processing.

ii. Processing of the image and development of the environment local map.

iii. Post processing algorithms to perform the following subtasks:

• Roundabout detection based on LS approach.

• Road following in the roads without curbs.

described in the following section.

3.1. Camera

134 Applications of Mobile Robots

main parts:

• Setting the brightness of live video

The brightness of image's sequences is adjusted, since the aperture of camera is not automatic. Following command is used:

set (vid.source, 'Brightness', 35);

• Start acquiring frames

This function helps to reserve the image device for the exclusive use in this program and locks the configuration to others applications. Thus, certain properties become read only while running.

Start(vid);

• Acquiring the image frames to MATLAB workspace

It returns the data, which contains the number of frames specified in the (Frames per Trigger) property of the video input object.

data = getdata(vid);

• Crop image

It allows for cutting the interesting regions in the images. In the proposed algorithm, the bottom half of the image is cropped, since it contains the nearest area of the road to the robot; also to save time needed for calculation in comparison with whole image.

```
GH = imcrop(data, [1 u1 io(2) io(1)]);
```
• Convert from RGB into grayscale

The purpose is to make image useful for the edge detection and removing the noise operations.

IS = rgb2gray(GH);

• Remove image acquisition object from memory

It is used to free memory at the end of an image acquisition session and start a new frame acquisition.

delete(vid).

(ii) Processing of the image and development of the environments local map.

It includes some operations that allow to extract the edges of the road and roundabout from the images with capability to remove the noises and perform filtrations.

The following operations are applied for edge detection and noise filtering:

1. 2D Gaussian filters: It is used for smoothing the detection of the edge. The point-spread function (PSF) for 2D continuous space Gaussian is given by:

$$\log(x,y) = \frac{1}{2\pi\sigma^2} \,\_2\text{er}^{-\left(\frac{\left(x-x\_0\right)^2}{2\sigma^2} + \frac{\left(y-y\_0\right)^2}{2\sigma^2}\right)}\tag{1}$$

It finds the edges using the Prewitt approximation of the derivative. Each component of the filter takes the derivative in one direction and smoothies in the orthogonal direction using a uniform filter. This filter is implemented with threshold coming from imfilter as follows:

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

BW = edge (I, 'prewitt', (graythresh(I) \* .1)).

It finds the edges using the Sobel approximation of the derivative. Each filter takes the derivative in one direction and smoothies in the orthogonal direction using a triangular

BW = edge (I, 'sobel', (graythresh(I) \* .1)).

The morphological methodology is applied to improve the shape of the lines representing the road curb, which are extracted from edges in the images using the above-mentioned filters. Morphology technique includes a various set of image processing enhancing operations that process the images based on shapes. Morphological operations implement a structuring element to the input image and then carry out the output images with a similar size. The morphological operation compares the current pixel in the output image with the input one and its neighbors to calculate the value of each output image pixels. This is done by selecting a certain shape and size of the neighbors that are sensitive to the input image's shapes.

Morphological structuring element (strel): it is used to define the areas that will be applied

se90 = strel('line', 3, 90). se0 = strel('line', 3, 0). Dilation of image (imdilate): dilation and erosion are two common operations in morphology. With dilation operation, the pixels in the image have to be summed to the object boundaries. The numbers of the added pixels are depended on the element size and its structure that is utilized to process the input images. In dilation process, the status of the pixels in the output images is figured out by implementing a certain rule and equations to the corresponding input pixels and all its neighbors. In the developed algorithm, the

BW1 = imdilate(BWC, [se90 se0]).

It is also called Min/Max of the Median filter. It uses a more general approach for filtration,

imdilate is used with a range that comes from the strel operation as follows:

which allows for specifying and choosing the rank order of the filter as:

and 90

are the shapes used for the

http://dx.doi.org/10.5772/intechopen.79412

137

filter. This filter is implemented with the threshold coming from imfilter as follows:

Two operations are used to perform the morphological method:

using morphological operations. Straight lines with 0

BWC is the binary image coming from the edges filters.

5. 2D order-statistic filtering (ordfil2)

images which actually represent the road curbs.

Sobel gradient filter:

4. Morphological operations

σ is the standard deviation of the Gaussian distribution.

The following Gaussian filter was applied to the image in MATLAB:

PSF = fspecial('gaussian', 3, 3).

2. Multidimensional images (imfilter): It allows the user to filter one array with another array according to the specified options. In the proposed image processing, a symmetric option is used to filter the image by the Gaussian filter as follows:

I = imfilter(IS, PSF, 'symmetric', 'conv').

IS is the gray value image of the scene, and conv indicates the linear convolution operation of the image in which each output pixel is the weighted sum of neighboring input pixels.

3. Canny, Prewitt, and Sobel filters for edge detection: these filters were used to find the edges of curbed road in the image. They are derivative-based operations for edge detection:

Canny filter: It finds the edges by looking for local maxima of the gradient of the image matrix. It passes through the following steps: smoothing the image by blurring the operation in order to prepare for removing the noise; finding the gradients of the edges, where the large magnitude gradients of the image should be marked; nonmaximum suppression operation which allow only the local maxima to be marked as edges; double thresholding applied to the potential edges; and edge tracking by hysteresis, where the final edges are determined by removing all edges that are not connected to certain strong edges (large magnitudes). This filter is implemented with threshold coming from imfilter as follows:

BW = edge (I, 'canny', (graythresh(I) \* .1)).

Prewitt gradient filter:

It finds the edges using the Prewitt approximation of the derivative. Each component of the filter takes the derivative in one direction and smoothies in the orthogonal direction using a uniform filter. This filter is implemented with threshold coming from imfilter as follows:

BW = edge (I, 'prewitt', (graythresh(I) \* .1)).

Sobel gradient filter:

• Remove image acquisition object from memory

acquisition. delete(vid).

136 Applications of Mobile Robots

It is used to free memory at the end of an image acquisition session and start a new frame

It includes some operations that allow to extract the edges of the road and roundabout from

1. 2D Gaussian filters: It is used for smoothing the detection of the edge. The point-spread

2. Multidimensional images (imfilter): It allows the user to filter one array with another array according to the specified options. In the proposed image processing, a symmetric option

I = imfilter(IS, PSF, 'symmetric', 'conv'). IS is the gray value image of the scene, and conv indicates the linear convolution operation of the image in which each output pixel is the weighted sum of neighboring input pixels.

3. Canny, Prewitt, and Sobel filters for edge detection: these filters were used to find the edges of curbed road in the image. They are derivative-based operations for edge detection:

Canny filter: It finds the edges by looking for local maxima of the gradient of the image matrix. It passes through the following steps: smoothing the image by blurring the operation in order to prepare for removing the noise; finding the gradients of the edges, where the large magnitude gradients of the image should be marked; nonmaximum suppression operation which allow only the local maxima to be marked as edges; double thresholding applied to the potential edges; and edge tracking by hysteresis, where the final edges are determined by removing all edges that are not connected to certain strong edges (large magnitudes). This filter is implemented with threshold coming from imfilter as

BW = edge (I, 'canny', (graythresh(I) \* .1)).

� <sup>x</sup>�<sup>x</sup> ð Þ<sup>0</sup> <sup>2</sup>

<sup>2</sup>σ<sup>2</sup> <sup>þ</sup> <sup>y</sup>�<sup>y</sup> ð Þ<sup>0</sup> <sup>2</sup> 2σ2 

(1)

1 <sup>2</sup>πσ<sup>2</sup> :<sup>e</sup>

(ii) Processing of the image and development of the environments local map.

the images with capability to remove the noises and perform filtrations.

function (PSF) for 2D continuous space Gaussian is given by:

σ is the standard deviation of the Gaussian distribution.

is used to filter the image by the Gaussian filter as follows:

PSF = fspecial('gaussian', 3, 3).

follows:

Prewitt gradient filter:

g xð Þ¼ ; y

The following Gaussian filter was applied to the image in MATLAB:

The following operations are applied for edge detection and noise filtering:

It finds the edges using the Sobel approximation of the derivative. Each filter takes the derivative in one direction and smoothies in the orthogonal direction using a triangular filter. This filter is implemented with the threshold coming from imfilter as follows:

BW = edge (I, 'sobel', (graythresh(I) \* .1)).

#### 4. Morphological operations

The morphological methodology is applied to improve the shape of the lines representing the road curb, which are extracted from edges in the images using the above-mentioned filters. Morphology technique includes a various set of image processing enhancing operations that process the images based on shapes. Morphological operations implement a structuring element to the input image and then carry out the output images with a similar size. The morphological operation compares the current pixel in the output image with the input one and its neighbors to calculate the value of each output image pixels. This is done by selecting a certain shape and size of the neighbors that are sensitive to the input image's shapes.

Two operations are used to perform the morphological method:

Morphological structuring element (strel): it is used to define the areas that will be applied using morphological operations. Straight lines with 0 and 90 are the shapes used for the images which actually represent the road curbs.

$$se90 = strel('line', \ 3, \ 90).$$

$$se0 = strel('line', \ 3, \ 0).$$

Dilation of image (imdilate): dilation and erosion are two common operations in morphology. With dilation operation, the pixels in the image have to be summed to the object boundaries. The numbers of the added pixels are depended on the element size and its structure that is utilized to process the input images. In dilation process, the status of the pixels in the output images is figured out by implementing a certain rule and equations to the corresponding input pixels and all its neighbors. In the developed algorithm, the imdilate is used with a range that comes from the strel operation as follows:

$$BW1 = 
intilde{(BWC, \(\{se90\,\,seO\})}.$$

BWC is the binary image coming from the edges filters.

5. 2D order-statistic filtering (ordfil2)

It is also called Min/Max of the Median filter. It uses a more general approach for filtration, which allows for specifying and choosing the rank order of the filter as:

$$f2 = 
ordleft[
\$1, 5, 
\; [1 
1 
1; 1 
1 
1; 1 
1]
].
$$

Where, the second parameter specifies the rank order chosen, and the third parameter specify the mask with the neighbors specified by the nonzero elements in it.

6. Removing of small objects from binary image (bwareaopen)

This function is helpful to remove all connected components (objects) that have less than a certain threshold numbers of pixels from the binary image. The connectivity between the pixels can be defined at any direction of image; in MATLAB, it can be defined by default to 4-connected neighborhood or 8-connected neighborhood for the images. In the proposed algorithm, it is adjusted as follows:

BW2 = bwareaopen(BW1, 1200).

7. Filling of image regions and hole operation (imfill)

Due to the object in images are not clear, especially at the borders, this function is used to fill the image region that represents an object. This tool is very useful in the proposed algorithm to detect the roundabout intersection. One can describe the region filling operation as follows:

$$X\_f = \left(X\_{f-1} \oplus B\right) \cap A^{\mathbb{C}} \quad f = 1, 2, 3, \dots \tag{2}$$

yLc ¼ Lc, xLc ¼ 0 yLa ¼ L<sup>a</sup> sin β

yLb ¼ L<sup>b</sup> sin β

<sup>¼</sup> Lc, xLc <sup>¼</sup> <sup>L</sup><sup>a</sup> cos <sup>β</sup>

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

, xLb <sup>¼</sup> <sup>L</sup><sup>b</sup> cos <sup>β</sup>

La, Lb, and Lc are the length of laser measurements at point a, b, and c, respectively, as shown in Figure 5. β is the angle between the measurement point and the platform coordinate system which can be calculated from the scan area angle L<sup>θ</sup> = (0–240�). yLa and yLc represent the

Figure 2. Image sequence processing where no roundabout can be detected, (a) original image, (b) curbed image in a

gray, (c) after edge detection algorithm, (d) final processed image, and (e) developed image for local map.

http://dx.doi.org/10.5772/intechopen.79412

(3)

139

Xf is any point inside the interested boundary, B is the symmetric structuring element, ∩ is the intersection area between A and B, and A<sup>C</sup> is the complement of set A. The abovementioned equation is performed in MATLAB by the following command:

$$P = \text{imfill}(\text{BV2}, \text{ 'holes'}).$$

The developed image sequences after processing looked like the one shown in Figures 2 and 3, which are used later on for Laser Simulator-based roundabout detection. Also, Figures 8–11 portrayed in Section 4 show the capability of the algorithm to develop the local map in the indoor and outdoor road environments.

#### 3.2. Laser range finder

LRF measurements are used to localize the robot in environments and for building 2D local map. As previously mentioned, this device can scan an area with 240� at 100 ms/scan. The surrounding environments in the LRF measurements look like the one shown in Figure 4.

Two coordinate systems are used to characterize the measurements of LRF as depicted in Figure 5, which are: LRF measuring coordinate and LRF environment coordinate systems. Since the LRF measurements are scanned starting from the origin point, the calculation is done based on the right triangle as depicted in Figure 5.

If the laser measurement of components in yL direction is compared at points a, b, and c, one can find:

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout http://dx.doi.org/10.5772/intechopen.79412 139

$$\begin{aligned} y\_{\rm Lc} &= L\_{\rm c}, & x\_{\rm Lc} &= 0\\ y\_{\rm La} &= L\_{\rm a} \sin \left( \beta \right) = L\_{\rm c}, & x\_{\rm Lc} &= L\_{\rm a} \cos \left( \beta \right) \\ y\_{\rm Lb} &= L\_{\rm b} \sin \left( \beta \right), & x\_{\rm Lb} &= L\_{\rm b} \cos \left( \beta \right) \end{aligned} \tag{3}$$

La, Lb, and Lc are the length of laser measurements at point a, b, and c, respectively, as shown in Figure 5. β is the angle between the measurement point and the platform coordinate system which can be calculated from the scan area angle L<sup>θ</sup> = (0–240�). yLa and yLc represent the

f2 = ordfilt2(x, 5, [111; 111; 111]). Where, the second parameter specifies the rank order chosen, and the third parameter

This function is helpful to remove all connected components (objects) that have less than a certain threshold numbers of pixels from the binary image. The connectivity between the pixels can be defined at any direction of image; in MATLAB, it can be defined by default to 4-connected neighborhood or 8-connected neighborhood for the images. In the proposed

BW2 = bwareaopen(BW1, 1200).

Due to the object in images are not clear, especially at the borders, this function is used to fill the image region that represents an object. This tool is very useful in the proposed algorithm to detect the roundabout intersection. One can describe the region filling oper-

Xf is any point inside the interested boundary, B is the symmetric structuring element, ∩ is the intersection area between A and B, and A<sup>C</sup> is the complement of set A. The above-

P = imfill(BW2, 'holes').

The developed image sequences after processing looked like the one shown in Figures 2 and 3, which are used later on for Laser Simulator-based roundabout detection. Also, Figures 8–11 portrayed in Section 4 show the capability of the algorithm to develop the

LRF measurements are used to localize the robot in environments and for building 2D local map. As previously mentioned, this device can scan an area with 240� at 100 ms/scan. The surrounding environments in the LRF measurements look like the one shown in Figure 4.

Two coordinate systems are used to characterize the measurements of LRF as depicted in Figure 5, which are: LRF measuring coordinate and LRF environment coordinate systems. Since the LRF measurements are scanned starting from the origin point, the calculation is done

If the laser measurement of components in yL direction is compared at points a, b, and c, one

mentioned equation is performed in MATLAB by the following command:

local map in the indoor and outdoor road environments.

based on the right triangle as depicted in Figure 5.

Xf <sup>¼</sup> Xf �<sup>1</sup> <sup>⊕</sup> <sup>B</sup> <sup>∩</sup> AC <sup>f</sup> <sup>¼</sup> <sup>1</sup>, <sup>2</sup>, <sup>3</sup>, :: (2)

specify the mask with the neighbors specified by the nonzero elements in it.

6. Removing of small objects from binary image (bwareaopen)

algorithm, it is adjusted as follows:

ation as follows:

138 Applications of Mobile Robots

3.2. Laser range finder

can find:

7. Filling of image regions and hole operation (imfill)

Figure 2. Image sequence processing where no roundabout can be detected, (a) original image, (b) curbed image in a gray, (c) after edge detection algorithm, (d) final processed image, and (e) developed image for local map.

Figure 3. Image sequence processing where roundabout is detected, (a) original image, (b) curbed image in a gray value, (c) with edge detection algorithm, (d) final image, and (e) developed image for local map.

parameters of the road, and yLb represents the curb. It is found that yLa and yLc have the same length in y direction; however, the length of yLb is different and can be written as follows:

$$y\_{\rm Lb} = y\_{\rm Lc} - \frac{Z\_{\rm Rb}}{\sin \left( \rho \right)} \Rightarrow y\_{\rm Lc} - y\_{\rm Lb} = \frac{Z\_{\rm Rb}}{\sin \left( \rho \right)} \tag{4}$$

yc � yd <sup>¼</sup> ZRd

yc � ye <sup>¼</sup> ZRe

The width of the obstacle can be calculated as:

Figure 5. Principle of laser measurement calculation.

sin ð Þ r

Figure 4. Laser measurements and the location of objects in its environment at the left, middle, and right side.

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

http://dx.doi.org/10.5772/intechopen.79412

141

sin ð Þ r

and xd ¼ -L<sup>d</sup> cos β

and x<sup>e</sup> ¼ -L<sup>e</sup> cos β

(5)

, (6)

Wob ¼ xe � xd (7)

ZRb = hc, h<sup>c</sup> is idetified as the road curbed height which is known as a threshold in the program. r is defined as the LRF rays and the floor. For obstacle detection, two successive scan measurements (i and ii) have to be compared with each other in yL direction in the location between e and d as illustrated in Figure 5 to find the obstacles ahead of the robot as shown in Eqs. (5)–(7):

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout http://dx.doi.org/10.5772/intechopen.79412 141

Figure 4. Laser measurements and the location of objects in its environment at the left, middle, and right side.

Figure 5. Principle of laser measurement calculation.

parameters of the road, and yLb represents the curb. It is found that yLa and yLc have the same length in y direction; however, the length of yLb is different and can be written as follows:

Figure 3. Image sequence processing where roundabout is detected, (a) original image, (b) curbed image in a gray value,

ZRb = hc, h<sup>c</sup> is idetified as the road curbed height which is known as a threshold in the program. r is defined as the LRF rays and the floor. For obstacle detection, two successive scan measurements (i and ii) have to be compared with each other in yL direction in the location between e and d as illustrated in Figure 5 to find the obstacles ahead of the robot as shown in Eqs. (5)–(7):

) <sup>y</sup>Lc � <sup>y</sup>Lb <sup>¼</sup> <sup>Z</sup>Rb

sin ð Þ <sup>r</sup> (4)

sin ð Þ r

<sup>y</sup>Lb <sup>¼</sup> <sup>y</sup>Lc � <sup>Z</sup>Rb

(c) with edge detection algorithm, (d) final image, and (e) developed image for local map.

140 Applications of Mobile Robots

$$y\_c - y\_d = \frac{Z\_{Rd}}{\sin\left(\rho\right)} \text{ and } \mathbf{x\_d} = \mathbf{-L\_d}\cos\left(\beta\right) \tag{5}$$

$$y\_{\varepsilon} - y\_{\varepsilon} = \frac{Z\_{R\varepsilon}}{\sin\left(\rho\right)} \text{and} \, \mathbf{x}\_{\varepsilon} = \cdot \mathbf{L}\_{\varepsilon} \cos\left(\beta\right) \,,\tag{6}$$

The width of the obstacle can be calculated as:

$$W\_{\alpha b} = \mathbf{x}\_{\varepsilon} - \mathbf{x}\_{d} \tag{7}$$

Figure 6. Laser measurements for the experimental setup, (a) one scan of laser measurement (mm), (b) road with curbs in 3D (mm), and (c) path generation (mm).

From previous calculation, one can define the parameters that will be used later for road discovering by LRF as shown in Figure 5 as:

• Road fluctuations (height of objects with reference to the laser device) as follows:

$$r\_{\text{ft}} = r\_n \cos \left(\beta \right) \tag{8}$$

encoders is 500 pulses/rotations, and the linear position of the wheels is calculated from the

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

<sup>C</sup> <sup>¼</sup> <sup>2</sup><sup>π</sup> r Pcur Pf<sup>r</sup>

C is the accumulative path of robot, r is the radius of wheel, Pcur is the number of pulses in the

Two encoders were used in the proposed system, and the accumulative path will be calculated

Cav <sup>¼</sup> <sup>C</sup><sup>1</sup> <sup>þ</sup> <sup>C</sup><sup>2</sup>

The proposed WMR is able to navigate autonomously on the road following and negotiates effectively the curvature of a standard roundabout. The robot is able to find the path starting from the predefined start pose until it reaches the pregoal position using the LS approach for the local map that was identified by the robot sensors such as camera, the LRF, and odometry measurements. Normally, the start and goal positions are determined using the DGPS system, and the robot uses this information to detect the direction of the roundabout exit. Since the proposed robot is navigating in a relatively small area (about 10–30 m), it is not useful to use GPS. Instead one can simply inform the robot about the roundabout exit direction, goal position before it starts to move. The goal position can be determined as how much distance the robot should travel after passing through the roundabout, for example, the goal is located at eastern part of the exit of the roundabout (i.e., 270�) with a distance equals to 20 m. The path that the robot will follow in the environment can be adjusted in the program of the navigation system as in middle, left or right. It is desired that the robot should navigate in the middle of the road for all the experiments. The velocity of the robot during navigation is adjusted in the

During navigation on the road, our Laser Simulator approach is used to detect the roundabout when it is present in the local map identified by the camera [27–29]. The sensor fusion including LRF and encoders is used to estimate the position of robot within the environments.

<sup>2</sup> (12)

http://dx.doi.org/10.5772/intechopen.79412

current position, and Pfr is the number of pulses for one full rotation of the encoder.

(11)

143

encoder rotation as shown in Figure 7 using the following expression:

as an average value of both encoders as follows:

Figure 7. Calculation of the linear position from the rotary encoder.

range of 7–10 cm/s.

4. Navigation system results and discussion

• Road width (side distance with reference to the laser device) as follows:

$$r\_{wn} = r\_n \sin\left(\beta\right) \tag{9}$$

rn is the dimension of LRF signals for n-th LRF measurements.

• Curb threshold: rf0 in β = 0 � is used as reference and the other fluctuation measurements (rfn) on the left and right side in the same scan are compared with this base line. If the deviation between the reference point and other measurement values exceeds the predefined threshold th, this point will be considered as a road curb. Otherwise, it will be considered as a road. This operation is repeated with all measurements as follows:

$$r\_{ft} - r\_{f0} \geq \pm t\_h \tag{10}$$

The LRF driver for reading the data from USB connection in real-time is developed in MATLAB. It includes some functions to identify, configure and get data from the sensors as further detailed out in Appendix B. The algorithm for detecting the curbs of road based on the previous mentioned Eqs. (3)–(10) is developed and implemented in the road environments as shown in Figure 6. The LRF is the main sensor for the calculation of the robot path within the environments, which produces the results through using the LRF to generate the equations for path planning as shown in Sections 4.1 and 4.2.

#### 3.3. Odometry

Two rotary encoders (B106) were used to estimate the position of robot, which are connected through dual brush card pins (IFC-BH02) to the two differential wheels. The full rotation of

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout http://dx.doi.org/10.5772/intechopen.79412 143

Figure 7. Calculation of the linear position from the rotary encoder.

From previous calculation, one can define the parameters that will be used later for road

Figure 6. Laser measurements for the experimental setup, (a) one scan of laser measurement (mm), (b) road with curbs in

rfn ¼ rn cos β

rwn ¼ rn sin β

(rfn) on the left and right side in the same scan are compared with this base line. If the deviation between the reference point and other measurement values exceeds the predefined threshold th, this point will be considered as a road curb. Otherwise, it will be considered as a road. This operation is repeated with all measurements as follows:

The LRF driver for reading the data from USB connection in real-time is developed in MATLAB. It includes some functions to identify, configure and get data from the sensors as further detailed out in Appendix B. The algorithm for detecting the curbs of road based on the previous mentioned Eqs. (3)–(10) is developed and implemented in the road environments as shown in Figure 6. The LRF is the main sensor for the calculation of the robot path within the environments, which produces the results through using the LRF to generate the equations for

Two rotary encoders (B106) were used to estimate the position of robot, which are connected through dual brush card pins (IFC-BH02) to the two differential wheels. The full rotation of

(8)

(9)

is used as reference and the other fluctuation measurements

rfn � rf <sup>0</sup> ≥ � th (10)

• Road fluctuations (height of objects with reference to the laser device) as follows:

• Road width (side distance with reference to the laser device) as follows:

rn is the dimension of LRF signals for n-th LRF measurements.

�

discovering by LRF as shown in Figure 5 as:

path planning as shown in Sections 4.1 and 4.2.

3.3. Odometry

• Curb threshold: rf0 in β = 0

3D (mm), and (c) path generation (mm).

142 Applications of Mobile Robots

encoders is 500 pulses/rotations, and the linear position of the wheels is calculated from the encoder rotation as shown in Figure 7 using the following expression:

$$C = \frac{2\pi \, r \, P\_{\text{cur}}}{P\_{\text{fr}}} \tag{11}$$

C is the accumulative path of robot, r is the radius of wheel, Pcur is the number of pulses in the current position, and Pfr is the number of pulses for one full rotation of the encoder.

Two encoders were used in the proposed system, and the accumulative path will be calculated as an average value of both encoders as follows:

$$\mathbb{C}\_{av} = \frac{\mathbb{C}\_1 + \mathbb{C}\_2}{2} \tag{12}$$

#### 4. Navigation system results and discussion

The proposed WMR is able to navigate autonomously on the road following and negotiates effectively the curvature of a standard roundabout. The robot is able to find the path starting from the predefined start pose until it reaches the pregoal position using the LS approach for the local map that was identified by the robot sensors such as camera, the LRF, and odometry measurements. Normally, the start and goal positions are determined using the DGPS system, and the robot uses this information to detect the direction of the roundabout exit. Since the proposed robot is navigating in a relatively small area (about 10–30 m), it is not useful to use GPS. Instead one can simply inform the robot about the roundabout exit direction, goal position before it starts to move. The goal position can be determined as how much distance the robot should travel after passing through the roundabout, for example, the goal is located at eastern part of the exit of the roundabout (i.e., 270�) with a distance equals to 20 m. The path that the robot will follow in the environment can be adjusted in the program of the navigation system as in middle, left or right. It is desired that the robot should navigate in the middle of the road for all the experiments. The velocity of the robot during navigation is adjusted in the range of 7–10 cm/s.

During navigation on the road, our Laser Simulator approach is used to detect the roundabout when it is present in the local map identified by the camera [27–29]. The sensor fusion including LRF and encoders is used to estimate the position of robot within the environments.

Figure 8. Image sequences processing where no roundabout can be detected using LS, (a) image from preprocessing and processing step, and (b) applying the LS (continues line in the middle).

Figure 9. Image sequences processing where roundabout is detected using LS, (a) image from preprocessing and processing step, and (b) applying the LS (discontinues line in the middle).

In general, two conditions are used to detect a roundabout; noncurbs detection on the left and right and discovering ellipse in the image as shown in Figures 8 and 9.

Figure 11 shows how the camera, encoders, and laser range finder are used for path planning and execution from start to goal position. It also shows the local map built from camera sequence frames, where the developed view presents the curbs of the road environment of road in images. The camera's local map is determined for each image in the sequences of video frames as shown in Figure 11(b) and (c), and the LS is applied to find the roundabout.

Figure 10. Correction of the robot path in the road following when the start location of the robot is near to left curb, (a) laser measurements, (b) path determination: the blue with (\*) is the laser measurements, black with (o) is road curbs and roundabout, and red with () is the optimum path, (c) images of the robot in the beginning of movement, and (d) images

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

http://dx.doi.org/10.5772/intechopen.79412

145

By applying the algorithm of cameras, laser range finder, and odometry described in Section 3, one can determine the path of robot as in Figure 11(d). It is noticed that the part of roundabout in the entrance and exit areas is not occurred in the roundabout environment, due to that the laser still does not reach that area, but it is calculated based on a mixed algorithm of camera and laser range finder. The road following curbs are shown in Figure 11(d), where the entrance to the roundabout is located on the left side, and the exit is located on the right. The path of the robot looked smooth especially with the indoor environments in comparison with the optimum path (continuous red line); however, there is a deviation in the outdoor measurements in

Figure 11(c) shows the last image, where the roundabout is detected.

of the robot when correcting its path.

#### 4.1. Road following

In the road following area, the camera, encoders, and laser are combined together to find the collision-free path. The camera is used for roundabout detection when it figures out using Laser Simulator. LRF is utilized for detecting the road curbs and localizing the robot in the environment. The encoder's measurements are used for estimating the current position of robot within environments. The generation of robot path is described in the following paragraphs.

#### 4.2. Roundabout navigation results

The results show the capability of the roundabout path planning algorithm to find the right trajectory with a small deviation in comparison to the optimum path as shown in Figure 10.

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout http://dx.doi.org/10.5772/intechopen.79412 145

Figure 10. Correction of the robot path in the road following when the start location of the robot is near to left curb, (a) laser measurements, (b) path determination: the blue with (\*) is the laser measurements, black with (o) is road curbs and roundabout, and red with () is the optimum path, (c) images of the robot in the beginning of movement, and (d) images of the robot when correcting its path.

In general, two conditions are used to detect a roundabout; noncurbs detection on the left and

Figure 9. Image sequences processing where roundabout is detected using LS, (a) image from preprocessing and

Figure 8. Image sequences processing where no roundabout can be detected using LS, (a) image from preprocessing and

In the road following area, the camera, encoders, and laser are combined together to find the collision-free path. The camera is used for roundabout detection when it figures out using Laser Simulator. LRF is utilized for detecting the road curbs and localizing the robot in the environment. The encoder's measurements are used for estimating the current position of robot within

The results show the capability of the roundabout path planning algorithm to find the right trajectory with a small deviation in comparison to the optimum path as shown in Figure 10.

environments. The generation of robot path is described in the following paragraphs.

right and discovering ellipse in the image as shown in Figures 8 and 9.

processing step, and (b) applying the LS (discontinues line in the middle).

processing step, and (b) applying the LS (continues line in the middle).

4.1. Road following

144 Applications of Mobile Robots

4.2. Roundabout navigation results

Figure 11 shows how the camera, encoders, and laser range finder are used for path planning and execution from start to goal position. It also shows the local map built from camera sequence frames, where the developed view presents the curbs of the road environment of road in images. The camera's local map is determined for each image in the sequences of video frames as shown in Figure 11(b) and (c), and the LS is applied to find the roundabout. Figure 11(c) shows the last image, where the roundabout is detected.

By applying the algorithm of cameras, laser range finder, and odometry described in Section 3, one can determine the path of robot as in Figure 11(d). It is noticed that the part of roundabout in the entrance and exit areas is not occurred in the roundabout environment, due to that the laser still does not reach that area, but it is calculated based on a mixed algorithm of camera and laser range finder. The road following curbs are shown in Figure 11(d), where the entrance to the roundabout is located on the left side, and the exit is located on the right. The path of the robot looked smooth especially with the indoor environments in comparison with the optimum path (continuous red line); however, there is a deviation in the outdoor measurements in

discovering and detecting the roundabout. A complete modeling and path determination of the roundabout intersection environments for autonomous mobile robot navigation is derived and presented in this chapter. The LS approach was used for road roundabout environment detection; whereas, the sensor fusion involves the laser range finder (LRF), and odometry was used to generate the path of the robot and localize it within its terrain. The local maps were built using both the camera and LRF to estimate the road border parameters such as road width, curbs, and roundabout in 2D. The algorithm of the path generation is fully derived within the local maps and subsequently implemented in two ways; first, considering the direct use of the sensor fusion data for path planning without applying localization and control algorithms and second, taking into account a complete model-based navigation including

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

http://dx.doi.org/10.5772/intechopen.79412

147

localization, path planning, and control schemes.

\* and Musa Mailah<sup>2</sup>

1 Faculty of Manufacturing Engineering, Universiti Malaysia Pahang, Pekan, Malaysia

2 Faculty of Mechanical Engineering, Universiti Teknologi Malaysia, Johor Bahru, Malaysia

[1] Ali MAH. Autonomous mobile robot navigation and control in the road following and roundabout environments incorporating laser range finder and vision system [PhD thesis].

[2] Ali MA, Mailah M, Yussof WA, Hamedon ZB, Yussof ZB, Majeed AP. Sensors fusion based online mapping and features extraction of mobile robot in the road following and roundabout. IOP Conference Series: Materials Science and Engineering. 2016;114(2016):1-12

[3] Miguel A. VIRTUOUS: Vision-based road transport-system for unmanned operation on urban-like scenarios. IEEE Transactions on Intelligent Transportation Systems. 2004;5(1):

[4] Matsushita Y, Miura J. On-line road boundary modeling with multiple sensory features, flexible road model and particle filter. Robotics and Autonomous Systems. 2011;59(1):274-284

[5] Seung H, Chi W, Sung C, Minyong P. Outdoor navigation of a mobile robot using differential GPS and curb detection. In: Proceedings of IEEE International Conference on

[6] Kazunori O, Takashi T, Bunji S, Shoichi M, Shin Y. Outdoor navigation of a mobile robot between buildings based on DGPS and Odometry data fusion. In: Proceedings of IEEE

Robotics and Automation; April 10–14. Roma, Italy: IEEE; 2007. pp. 3414-3419

\*Address all correspondence to: hashem@ump.edu.my

Author details

References

UTM; 2014

69-83

Mohammed A. H. Ali1

Figure 11. Camera sequence images when navigating in outdoor with 360 rotation, (a) image when start moving, (b) camera's local map when robot starts to move, (c) camera's local map when robot detects the roundabout, and (d) path of robot during navigation in a roundabout in (mm) with 360 rotation: blue with (\*) is the path, black with (O) is for the road environments, and red with () for the optimum path.

the area of entrance and exit of roundabout due to the distance between the curbs and the roundabout center.

#### 5. Conclusion

A practical implementation of the WMR online path navigation using laser simulator (LS) and sensor fusion in the road environments is presented in this thesis. The mobile robot is supposed to navigate on the roads in real time and reach the predetermined goal while discovering and detecting the roundabout. A complete modeling and path determination of the roundabout intersection environments for autonomous mobile robot navigation is derived and presented in this chapter. The LS approach was used for road roundabout environment detection; whereas, the sensor fusion involves the laser range finder (LRF), and odometry was used to generate the path of the robot and localize it within its terrain. The local maps were built using both the camera and LRF to estimate the road border parameters such as road width, curbs, and roundabout in 2D. The algorithm of the path generation is fully derived within the local maps and subsequently implemented in two ways; first, considering the direct use of the sensor fusion data for path planning without applying localization and control algorithms and second, taking into account a complete model-based navigation including localization, path planning, and control schemes.

## Author details

Mohammed A. H. Ali1 \* and Musa Mailah<sup>2</sup>

\*Address all correspondence to: hashem@ump.edu.my


#### References

the area of entrance and exit of roundabout due to the distance between the curbs and the

Figure 11. Camera sequence images when navigating in outdoor with 360 rotation, (a) image when start moving, (b) camera's local map when robot starts to move, (c) camera's local map when robot detects the roundabout, and (d) path of robot during navigation in a roundabout in (mm) with 360 rotation: blue with (\*) is the path, black with (O) is for the

A practical implementation of the WMR online path navigation using laser simulator (LS) and sensor fusion in the road environments is presented in this thesis. The mobile robot is supposed to navigate on the roads in real time and reach the predetermined goal while

roundabout center.

146 Applications of Mobile Robots

road environments, and red with () for the optimum path.

5. Conclusion


International Conference on Robotics and Automation; September 14–19. Taipei, Taiwan: IEEE; 2003. pp. 1978-1984

[19] Campbel N, Pout M, Priestly M, Dagless E, Thomas BT. Autonomous road vehicle navi-

Online Mapping-Based Navigation System for Wheeled Mobile Robot in Road Following and Roundabout

http://dx.doi.org/10.5772/intechopen.79412

149

[20] Jing-Chuan W, Shiyu H, Zhang X, Weidong C. Mobile robot localization in outdoor environments based on infrared vision. In: International Conference on Intelligent Computing for Sustainable Energy and Environment LSMS/ ICSEE 2010. September 17–20,

[21] Locchi L, Kanolige K. Visually realistic mapping of a planar environment with stereo. In: Proceedings of Seventh International Symposium on Experimental Robotics (ISER).

[22] Castellanos J, Martfinez M, Neira J, Tadus D. Simultaneous map building and localization for mobile robot: A multi-sensor fusion approach. In: Proceedings of IEEE International Conference on Robotics and Automation. May 16–20, 1998. Leuven, Belgium: IEEE; 1998.

[23] Leonimer F, Jose F. Trajectory planning for non-holonomic mobile robot using extended Kalman filter. Mathematical Problems in Engineering. 2010;2010:979205, 22 pp

[24] Hong T, Christopher R, Tommy C, Michael S. Road detection and tracking for autonomous mobile robots. In: Proceedings of the SPIE 16th Annual International Symposium on Aerospace/Defense, Sensing, Simulation and controls. April 1–5. Orlando, FL: SPIE; 2002.

[25] Aneesh C, Yuta S. Road-crossing landmarks detection by outdoor mobile robots. Journal

[26] Labayrade R, Royere C, Gruyer D, Aubert D. Cooperative fusion for multi-obstacles detection with use of stereovision and laser scanner. Autonomous Robots. 2005;19(2):117-140 [27] Ali M, Mailah M, Tang HH. A novel approach for visibility search graph based path planning. In: 13th Int. Conf. Robot., Control Manuf. Syst.; Kuala-Lumpur, Malaysia, April

[28] Ali M, Mailah M, Tang HH. Implementation of laser simulator search graph for detection and path planning in roundabout environments. WSEAS Transactions on Signal

[29] Ali MAH, Yusoff WABW, Hamedon ZB, Yusssof ZBM, Mailah M. Global mobile robot path planning using laser simulator. In: 2nd IEEE International on Robotics and

gation. Engineering Applications of Artificial Intelligence. 1994;7(2):177-190

December 10–13, 2000. Honolulu, Hawaii: Springer; 2000. pp. 521-532

2010. Wuxi, China: Springer-Verlag; 2010. pp. 33-41

of Robotics and Mechatronics. 2010;22(6):708-717

Manufacturing Automation (ROMA); Ipoh: Malaysia; 2016

pp. 1244-1249

pp. 1-8

2–4, 2013. pp. 44-49

Processing. 2014;10(2014):118-126


[19] Campbel N, Pout M, Priestly M, Dagless E, Thomas BT. Autonomous road vehicle navigation. Engineering Applications of Artificial Intelligence. 1994;7(2):177-190

International Conference on Robotics and Automation; September 14–19. Taipei, Taiwan:

[7] Jose G, Eduardo N, Stephan B. Autonomous navigation and map building using laser range sensors in outdoor applications. Journal of Robotic Systems. 2000;17(10):565-583

[8] Donghoon L, Sukyung S, Kwangwoong Y, Jonghwan P, Hogil L. Sensor fusion localization system for outdoor mobile robot. In: ICROS-SICE International Joint Conference;

[9] Cristina C, Dolores B, Luis M. Compact modeling technique for outdoor navigation. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans. 2008;

[10] Florian A, Franz A, Sven L, Lukas G, Joerg D. An unmanned helicopter for autonomous flights in urban terrain. In: German Workshop on Robotics; June 9–10. Braunschweig,

[11] Panzieri S, Pascucci F, Ulivi G. An outdoor navigation system using GPS and inertial platform. In: Proceedings of IEEE/ASME International Conference on Advanced Intelli-

[12] Thrapp R, Westbrook C, Subramanian D. Robust localization algorithms for an autonomous campus tour guide. In: Proceedings of IEEE International Conference on Robotics

[13] Kazuyuki K, Cheok K, Kajiro W, Fumio M. Accurate differential global positioning system via fuzzy logic Kalman filter sensor fusion technique. IEEE Transactions on Industrial

[14] Cherif S, El-Najjar M, François C. Multi-sensor fusion method using dynamic Bayesian network for precise vehicle localization and road matching. In: Proceedings of 19th IEEE International Conference on Tools with Artificial Intelligence. October 29–31, 2007. Patras,

[15] Chao G, Michael S, John R. Towards autonomous wheelchair systems in urban environments. In: Howard A et al., editors. Field and Service Robotics. Vol. 7. Heidelberg-Berlin:

[16] Goldbeck J, Huertgen B, Ernsta S, Kelch L. Lane following combining vision and DGPS.

[17] Nebot E, Sukkarieh S, Hugh D. Inertial navigation aided with GPS information. In: IEEE Conference on Mechatronics and Machine Vision in Practice. September 23–25, 1997.

[18] Eric N, Jacques G, Tarbouchi M, Umar I, Aboelmagd, N. Enhanced mobile robot outdoor localization using INS/GPS integration. In: International Conference on Computer Engineering and Systems ICCES 2009. December 14–16, 2009. Cairo, Egypt: IEEE Xplore; 2009.

and Automation. May 21–26, 2001. Seoul, Korea: IEEE; 2001. pp. 2065-2071

August 18–21, 2009. Fukuoka, Japan: SICE; 2009. pp. 1384-1387

Germany: Aerospace Center (DLR). pp. 1-11

Electronics. 1998;45(3):510-528

Greece: IEEE; 2007. pp. 146-151

Springer-Verlag; 2010. pp. 13-23

pp. 127-132

gent Mechatronics: IEEE/ASME; 2001. pp. 1346-1351

Journal of Image and Vision Computing. 2000;18:425-433

Toowoomba, Australia: IEEE; 1997. pp. 169-174

IEEE; 2003. pp. 1978-1984

38(1):9-24

148 Applications of Mobile Robots


**Chapter 8**

Provisional chapter

**Path Tracking of a Wheeled Mobile Manipulator**

DOI: 10.5772/intechopen.79598

Path Tracking of a Wheeled Mobile Manipulator

**through Improved Localization and Calibration**

This chapter focuses on path tracking of a wheeled mobile manipulator designed for manufacturing processes such as drilling, riveting, or line drawing, which demand high accuracy. This problem can be solved by combining two approaches: improved localization and improved calibration. In the first approach, a full-scale kinematic equation is derived for calibration of each individual wheel's geometrical parameters, as opposed to traditionally treating them identical for all wheels. To avoid the singularity problem in computation, a predefined square path is used to quantify the errors used for calibration considering the movement in different directions. Both statistical method and interval analysis method are adopted and compared for estimation of the calibration parameters. In the second approach, a vision-based deviation rectification solution is presented to localize the system in the global frame through a number of artificial reflectors that are identified by an onboard laser scanner. An improved tracking and localization algorithm is developed to meet the high positional accuracy requirement, improve the system's repeatability in the traditional trilateral algorithm, and solve the problem of pose loss in path following. The developed methods have been verified and implemented on the

Recently, mobile manipulators have been used in various industries including aerospace or indoor decoration engineering, which requires a large workspace [1–4]. The said mobile manipulator consists of an industrial manipulator mounted on a mobile platform to perform various manufacturing tasks such as drilling/riveting in aerospace industry or baseline

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

through Improved Localization and Calibration

Tao Song, Fengfeng (Jeff) Xi and Shuai Guo

Tao Song, Fengfeng (Jeff) Xi and Shuai Guo

Additional information is available at the end of the chapter

mobile manipulators developed by Shanghai University.

Keywords: mobile manipulator, localization, tracking, path following

Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79598

Abstract

1. Introduction

#### **Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration** Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

DOI: 10.5772/intechopen.79598

Tao Song, Fengfeng (Jeff) Xi and Shuai Guo Tao Song, Fengfeng (Jeff) Xi and Shuai Guo

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79598

#### Abstract

This chapter focuses on path tracking of a wheeled mobile manipulator designed for manufacturing processes such as drilling, riveting, or line drawing, which demand high accuracy. This problem can be solved by combining two approaches: improved localization and improved calibration. In the first approach, a full-scale kinematic equation is derived for calibration of each individual wheel's geometrical parameters, as opposed to traditionally treating them identical for all wheels. To avoid the singularity problem in computation, a predefined square path is used to quantify the errors used for calibration considering the movement in different directions. Both statistical method and interval analysis method are adopted and compared for estimation of the calibration parameters. In the second approach, a vision-based deviation rectification solution is presented to localize the system in the global frame through a number of artificial reflectors that are identified by an onboard laser scanner. An improved tracking and localization algorithm is developed to meet the high positional accuracy requirement, improve the system's repeatability in the traditional trilateral algorithm, and solve the problem of pose loss in path following. The developed methods have been verified and implemented on the mobile manipulators developed by Shanghai University.

Keywords: mobile manipulator, localization, tracking, path following

#### 1. Introduction

Recently, mobile manipulators have been used in various industries including aerospace or indoor decoration engineering, which requires a large workspace [1–4]. The said mobile manipulator consists of an industrial manipulator mounted on a mobile platform to perform various manufacturing tasks such as drilling/riveting in aerospace industry or baseline

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

drawing in decoration engineering. Wheeled mobile platforms with Mecanum wheels that can easily navigate through crowded spaces due to their omnidirectionality with a zero turning radius are commonly used. Path tracking is one of the important issues for mobile manipulators, in particular for performing manufacturing tasks. This chapter addresses this issue from the aspect of localization and calibration.

Localization is a key functionality of mobile manipulator in order to track and determine its position around the environment [5]. Many methods are proposed to address this issue [6]. They can be divided into two categories: absolute localization and relative localization.

Absolute localization relies on detecting and recognizing different features in the environment to obtain the position and posture. The features can be normally divided into two types: artificial landmarks and natural landmarks. Compared with the natural landmarks, artificial landmarks have advantages of high recognition, which will lead to high accuracy. There is no cumulative error problem when a localization method based on artificial landmarks is used. The key challenge is to identify and extract the needed information from the raw data of landmarks. For the relative localization, dead reckoning and inertial navigation are commonly carried out to obtain the systems' position. It does not have to perceive the external environment, but the drift error accumulates over time.

Researchers have proposed several solutions, such as fuzzy reflector-based localization [7] and color reflector-based self-localization [8]. The main drawbacks of these two methods are that the anti-interference ability is poor and the computation is huge. To solve these problems pertaining to the localization method based on artificial landmarks, Madsen and Andersen [9] proposed a method using three reflectors and the triangulation principle. Betke and Gurvits [10] proposed a multichannel localization method with the three-dimensional localization principle and the least squares method. Because of the unavoidable errors in position and angle measurement of reflectors, the use of only a trilateral or triangular method will not achieve high accuracy [11]. Nevertheless, there are still many challenges for mobile manipulator working in industrial environments such as aerospace manufacturing or decoration engineering, which requires high maneuverability and high accuracy at the same time. The stationary industrial manipulator has high repeated localization accuracy, which the mobile manipulator cannot achieve. This chapter focuses on the improvement of path tracking of a mobile manipulator through enhanced localization combined with calibration. Calibration is required for the system kinematic model established based on nominal geometry parameter to improve motion accuracy. Muir and Neuman [12] proposed a kinematic error model for Mecanum wheeled platform and applied actuated inverse and sensed forward solutions to the kinematic control. Wang and Chang [13] carried out error analysis in terms of distribution among Mecanum wheels. Shimada et al. [14] presented a position-corrective feedback control method with vision system on Mecanum wheeled mobile platform. Qian et al. [15] conducted a more detailed analysis on the installation angle of rollers. An improved calibration method is presented in this chapter to improve the tracking accuracy of a mobile manipulator.

#### 2. System modeling

The wheeled mobile manipulator, as shown in Figure 1, is built with a manipulator onto a wheeled mobile platform with four Mecanum wheels. This system aims to carry out fuselage drilling/riveting tasks at assembly stations in an adaptive and flexible way in the aero-

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

http://dx.doi.org/10.5772/intechopen.79598

153

Figure 1. The wheeled mobile manipulator for fuselage drilling/riveting.

This system needs to localize in real time during machining process. The global frame f g XG, YG, ZG is attached to the ground in the environment. The platform frame Xf g <sup>P</sup>, YP, ZP is

space industry.

Figure 2. The position and posture of system.

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration http://dx.doi.org/10.5772/intechopen.79598 153

Figure 1. The wheeled mobile manipulator for fuselage drilling/riveting.

drawing in decoration engineering. Wheeled mobile platforms with Mecanum wheels that can easily navigate through crowded spaces due to their omnidirectionality with a zero turning radius are commonly used. Path tracking is one of the important issues for mobile manipulators, in particular for performing manufacturing tasks. This chapter addresses this issue from

Localization is a key functionality of mobile manipulator in order to track and determine its position around the environment [5]. Many methods are proposed to address this issue [6]. They can be divided into two categories: absolute localization and relative localization.

Absolute localization relies on detecting and recognizing different features in the environment to obtain the position and posture. The features can be normally divided into two types: artificial landmarks and natural landmarks. Compared with the natural landmarks, artificial landmarks have advantages of high recognition, which will lead to high accuracy. There is no cumulative error problem when a localization method based on artificial landmarks is used. The key challenge is to identify and extract the needed information from the raw data of landmarks. For the relative localization, dead reckoning and inertial navigation are commonly carried out to obtain the systems' position. It does not have to perceive the external environ-

Researchers have proposed several solutions, such as fuzzy reflector-based localization [7] and color reflector-based self-localization [8]. The main drawbacks of these two methods are that the anti-interference ability is poor and the computation is huge. To solve these problems pertaining to the localization method based on artificial landmarks, Madsen and Andersen [9] proposed a method using three reflectors and the triangulation principle. Betke and Gurvits [10] proposed a multichannel localization method with the three-dimensional localization principle and the least squares method. Because of the unavoidable errors in position and angle measurement of reflectors, the use of only a trilateral or triangular method will not achieve high accuracy [11]. Nevertheless, there are still many challenges for mobile manipulator working in industrial environments such as aerospace manufacturing or decoration engineering, which requires high maneuverability and high accuracy at the same time. The stationary industrial manipulator has high repeated localization accuracy, which the mobile manipulator cannot achieve. This chapter focuses on the improvement of path tracking of a mobile manipulator through enhanced localization combined with calibration. Calibration is required for the system kinematic model established based on nominal geometry parameter to improve motion accuracy. Muir and Neuman [12] proposed a kinematic error model for Mecanum wheeled platform and applied actuated inverse and sensed forward solutions to the kinematic control. Wang and Chang [13] carried out error analysis in terms of distribution among Mecanum wheels. Shimada et al. [14] presented a position-corrective feedback control method with vision system on Mecanum wheeled mobile platform. Qian et al. [15] conducted a more detailed analysis on the installation angle of rollers. An improved calibration method is presented in this chapter to improve the tracking accuracy of a mobile manipulator.

The wheeled mobile manipulator, as shown in Figure 1, is built with a manipulator onto a wheeled mobile platform with four Mecanum wheels. This system aims to carry out fuselage

the aspect of localization and calibration.

152 Applications of Mobile Robots

ment, but the drift error accumulates over time.

2. System modeling

Figure 2. The position and posture of system.

drilling/riveting tasks at assembly stations in an adaptive and flexible way in the aerospace industry.

This system needs to localize in real time during machining process. The global frame f g XG, YG, ZG is attached to the ground in the environment. The platform frame Xf g <sup>P</sup>, YP, ZP is attached to the center of the mobile platform. The tool frame Xf g <sup>M</sup>;YM;ZM is attached to the tool of the manipulator. Two laser range finders are equipped and their frames Xf g L1;YL1;ZL1 and Xf g L2;YL2;ZL2 are attached to the left-front and right-rear of the mobile platform, respectively. The vision frame Xf g <sup>V</sup>;YV;ZV is attached to the industrial camera.

The position and posture of the system in the global frame can be defined as

$$\mathbf{^G P} = \begin{bmatrix} \mathbf{^G x} & \mathbf{^G y} & \mathbf{^G} \end{bmatrix} \tag{1}$$

where V ¼ ½ � VX; VY; ω<sup>0</sup>

ground and the wheel.

where <sup>D</sup><sup>1</sup> <sup>¼</sup> <sup>1</sup>

4

Figure 4. Mecanum wheel and roller.

R1 l1 <sup>þ</sup> <sup>L</sup> � R2

½ � VX; VY; ω<sup>0</sup>

T , D0 <sup>¼</sup> <sup>1</sup> 4

the roller, Eq. (2) can be revised as follows: [18].

R0 l0 <sup>þ</sup> <sup>L</sup> � R0

ω3, and ω<sup>4</sup> are angular velocities of the four wheels, respectively.

�R1 R2 �R3 R4 R1 R2 R3 R4

l2 <sup>þ</sup> <sup>L</sup> � R3

l3 þ L

between li and l0 (i ¼ 1, 2, 3, 4) should be obtained to revise matrix D<sup>1</sup> (Figure 6).

�R0 R0 �R0 R0 R0 R0 R0 R0

l0 <sup>þ</sup> <sup>L</sup> � R0

from the front axle to the rear axle as shown in Figure 3; l0 is the transverse distance from the wheel centers to the platform center line; R0 is the radius of the Mecanum wheel; and ω1, ω2,

The Mecanum wheel and its roller are shown in Figure 4. The roller is fitted on the edge of the wheel at a certain angle (generally 45�) with its central axis. Each roller can rotate freely around the central axis. The wheel relies on the friction between the roller and the ground to move. The material of roller's outer rim is usually rubber, which will deform under pressure between

Figure 5 shows the distribution of roller deformation. F and T are the force and driving torque on the roller, respectively. The radiuses of wheels reduce differently under different pressures. The deformation zone and the position of the wheel center change with the action of driving torque and the shifting [17]. Furthermore, with such deformation, R0 and l0 will change.

As shown in Figure 6, based on the kinematic analysis and consideration on deformation of

R4 l4 þ L

which are individual radiuses of the four Mecanum wheels, respectively. In order to improve the system motion accuracy, the relative errors ΔRi between Ri and R0, and relative errors Δli

<sup>T</sup> is defined as the linear and angular speeds of the platform; L is the half distance

l0 þ L

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

R0 l0 þ L

V ¼ D1W (3)

; here D<sup>1</sup> is defined by R1, R2, R3, and R4,

3 7 7

5, <sup>W</sup> <sup>¼</sup> ½ � <sup>ω</sup>1; <sup>ω</sup>2; <sup>ω</sup>3; <sup>ω</sup><sup>4</sup>

http://dx.doi.org/10.5772/intechopen.79598

T . 155

where <sup>G</sup>x and <sup>G</sup>y are the positions in two directions, respectively, and <sup>G</sup>θ is the azimuth angle, as shown in Figure 2.

#### 3. Accuracy analysis of a wheeled mobile manipulator

#### 3.1. Problem formulation

Figure 3 shows the kinematic model of the mobile platform with Mecanum wheels. According to kinematic analysis [16], the motion equation can be obtained as

$$\mathbf{V} = \mathbf{D}\_0 \mathbf{W} \tag{2}$$

Figure 3. The kinematic model of a Mecanum wheeled mobile platform.

$$\text{where } \mathbf{V} = \begin{bmatrix} \mathbf{V}\chi, \mathbf{V}\_{\mathbf{I}}, \omega\boldsymbol{\omega} \end{bmatrix}^{\mathrm{T}}, \mathbf{D}\_{\mathbf{0}} = \frac{1}{4} \begin{bmatrix} -\mathbf{R}\_{0} & \mathbf{R}\_{0} & -\mathbf{R}\_{0} & \mathbf{R}\_{0} \\ \mathbf{R}\_{0} & \mathbf{R}\_{0} & \mathbf{R}\_{0} & \mathbf{R}\_{0} \\ \frac{\mathbf{R}\_{0}}{\mathbf{I}\_{0} + \mathbf{L}} & -\frac{\mathbf{R}\_{0}}{\mathbf{I}\_{0} + \mathbf{L}} & -\frac{\mathbf{R}\_{0}}{\mathbf{I}\_{0} + \mathbf{L}} & \frac{\mathbf{R}\_{0}}{\mathbf{I}\_{0} + \mathbf{L}} \end{bmatrix}, \mathbf{W} = \begin{bmatrix} \omega\mathbf{I}\_{1}, \omega\mathbf{I}\_{2}, \omega\mathbf{u}, \omega\mathbf{u} \end{bmatrix}^{\mathrm{T}}.$$

½ � VX; VY; ω<sup>0</sup> <sup>T</sup> is defined as the linear and angular speeds of the platform; L is the half distance from the front axle to the rear axle as shown in Figure 3; l0 is the transverse distance from the wheel centers to the platform center line; R0 is the radius of the Mecanum wheel; and ω1, ω2, ω3, and ω<sup>4</sup> are angular velocities of the four wheels, respectively.

The Mecanum wheel and its roller are shown in Figure 4. The roller is fitted on the edge of the wheel at a certain angle (generally 45�) with its central axis. Each roller can rotate freely around the central axis. The wheel relies on the friction between the roller and the ground to move. The material of roller's outer rim is usually rubber, which will deform under pressure between ground and the wheel.

Figure 5 shows the distribution of roller deformation. F and T are the force and driving torque on the roller, respectively. The radiuses of wheels reduce differently under different pressures. The deformation zone and the position of the wheel center change with the action of driving torque and the shifting [17]. Furthermore, with such deformation, R0 and l0 will change.

As shown in Figure 6, based on the kinematic analysis and consideration on deformation of the roller, Eq. (2) can be revised as follows: [18].

$$\mathbf{V} = \mathbf{D}\_1 \mathbf{W} \tag{3}$$

where <sup>D</sup><sup>1</sup> <sup>¼</sup> <sup>1</sup> 4 �R1 R2 �R3 R4 R1 R2 R3 R4 R1 l1 <sup>þ</sup> <sup>L</sup> � R2 l2 <sup>þ</sup> <sup>L</sup> � R3 l3 þ L R4 l4 þ L 2 6 6 4 3 7 7 5 ; here D<sup>1</sup> is defined by R1, R2, R3, and R4,

which are individual radiuses of the four Mecanum wheels, respectively. In order to improve the system motion accuracy, the relative errors ΔRi between Ri and R0, and relative errors Δli between li and l0 (i ¼ 1, 2, 3, 4) should be obtained to revise matrix D<sup>1</sup> (Figure 6).

Figure 4. Mecanum wheel and roller.

attached to the center of the mobile platform. The tool frame Xf g <sup>M</sup>;YM;ZM is attached to the tool of the manipulator. Two laser range finders are equipped and their frames Xf g L1;YL1;ZL1 and Xf g L2;YL2;ZL2 are attached to the left-front and right-rear of the mobile platform, respec-

where <sup>G</sup>x and <sup>G</sup>y are the positions in two directions, respectively, and <sup>G</sup>θ is the azimuth

Figure 3 shows the kinematic model of the mobile platform with Mecanum wheels. According

<sup>G</sup><sup>P</sup> <sup>¼</sup> Gx, Gy, <sup>G</sup><sup>θ</sup> (1)

V ¼ D0W (2)

tively. The vision frame Xf g <sup>V</sup>;YV;ZV is attached to the industrial camera. The position and posture of the system in the global frame can be defined as

3. Accuracy analysis of a wheeled mobile manipulator

to kinematic analysis [16], the motion equation can be obtained as

Figure 3. The kinematic model of a Mecanum wheeled mobile platform.

angle, as shown in Figure 2.

154 Applications of Mobile Robots

3.1. Problem formulation

<sup>X</sup> <sup>¼</sup> <sup>1</sup> 4

leading to

where <sup>Δ</sup><sup>X</sup> <sup>¼</sup> ½ � <sup>Δ</sup>X; <sup>Δ</sup>Y; <sup>Δ</sup><sup>θ</sup> <sup>T</sup>

G ¼

li and Ri can be defined as:

posture <sup>G</sup>Xk ;

calculated as

<sup>G</sup>Yk ; <sup>G</sup>θ<sup>k</sup>

ΔX ¼

θ1 M1

R1

�θ<sup>1</sup> θ<sup>2</sup> �θ<sup>3</sup> θ<sup>1</sup> θ<sup>2</sup> θ<sup>3</sup>

> � <sup>θ</sup><sup>2</sup> M2

� <sup>θ</sup><sup>3</sup> M3

With the least squares method, the geometric errors can be solved as

l1 <sup>þ</sup> <sup>L</sup> <sup>θ</sup><sup>1</sup> � R2

�R1θ<sup>1</sup> R2θ<sup>2</sup> �R3θ<sup>3</sup> R4θ<sup>4</sup> R1θ<sup>1</sup> R2θ<sup>2</sup> R3θ<sup>3</sup> R4θ<sup>4</sup>

l2 <sup>þ</sup> <sup>L</sup> <sup>θ</sup><sup>2</sup> � R3

, ΔB ¼ ½ � ΔR1; ΔR2; ΔR3; ΔR4; Δl1; Δl2; Δl3; Δl4

θ4 M4

<sup>Δ</sup><sup>B</sup> <sup>¼</sup> <sup>G</sup><sup>T</sup><sup>G</sup> � ��<sup>1</sup>

R1 ¼ R0 þ ΔR1, l1 ¼ l0 þ Δl1, R2 ¼ R0 þ ΔR2, l2 ¼ l0 þ Δl2

R3 ¼ R0 þ ΔR3, l3 ¼ l0 þ Δl3, R4 ¼ R0 þ ΔR4, l4 ¼ l0 þ Δl4

ΔRi is generally defined as the tolerance resulting from manufacturing or assembly. With the deformation of the roller and Δli limited to ½ � –h; h and ½ � –j; j ðh ¼ 3mm, j ¼ 5mm) respectively,

where li and Ri are in l½ � <sup>0</sup> � j; l0 þ j and R½ � <sup>0</sup> � h;R0 þ h , respectively, and r is a random number between 0 and 1. The angular speeds of all wheels are set to 20rad=s, and the time t is set to 1 s. The rank of matrix G in Eq. (7) is generally 3, so G is a full rank matrix and ΔB can be obtained. The following steps can be carried out. First, the displacement error measurement is analyzed.

The system is moved in the YG direction and stopped every 2 s to obtain its position and

<sup>G</sup>X1 <sup>þ</sup> <sup>G</sup><sup>X</sup> <sup>2</sup> <sup>þ</sup> <sup>G</sup><sup>X</sup> <sup>3</sup> <sup>þ</sup> <sup>G</sup><sup>X</sup> <sup>4</sup> � �=<sup>10</sup> � XT

<sup>G</sup><sup>Y</sup> <sup>1</sup> <sup>þ</sup> <sup>G</sup><sup>Y</sup> <sup>2</sup> <sup>þ</sup> <sup>G</sup><sup>Y</sup> <sup>3</sup> <sup>þ</sup> <sup>G</sup><sup>Y</sup> <sup>4</sup> � �=<sup>10</sup> � YT

<sup>G</sup><sup>θ</sup> <sup>1</sup> <sup>þ</sup> <sup>G</sup><sup>θ</sup> <sup>2</sup> <sup>þ</sup> <sup>G</sup><sup>θ</sup> <sup>3</sup> <sup>þ</sup> <sup>G</sup><sup>θ</sup> <sup>4</sup> � �=<sup>10</sup> � <sup>θ</sup><sup>T</sup>

� �. The principle of measurement is shown in Figure 7 and ΔX can be

In order to obtain ΔB, it is needed to obtain the displacement error matrix ΔX first.

θ<sup>4</sup> 0 0 θ<sup>4</sup> 0 0

> � R1θ<sup>1</sup> M1 2

l3 <sup>þ</sup> <sup>L</sup> <sup>θ</sup><sup>3</sup>

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

R2θ<sup>2</sup> M2 2

R4 l4 <sup>þ</sup> <sup>L</sup> <sup>θ</sup><sup>4</sup>

ΔX ¼ GΔB (6)

T , and

0 0

0 0

G<sup>T</sup>ΔX (7)

li ¼ 2jr þ ð Þ l0 � j (8)

Ri ¼ 2hr þ ð Þ R0 � h (9)

(10)

<sup>2</sup> � R4θ<sup>4</sup> M4 2

R3θ<sup>3</sup> M3

http://dx.doi.org/10.5772/intechopen.79598

(5)

157

Figure 5. Roller deformation.

Figure 6. Motion model of the mobile platform.

#### 3.2. Error modeling

By multiplying time t, Eq. (3) becomes a displacement equation as

$$\mathbf{X} = \mathbf{D}\_1 \boldsymbol{\Theta} \tag{4}$$

where <sup>X</sup> <sup>¼</sup> <sup>V</sup>t is X½ � ;Y; <sup>θ</sup> <sup>T</sup> . θ ¼ Wt is ½ � θ1; θ2; θ3; θ<sup>4</sup> T . Individual geometric parameters including l1, l2, l3, l4, R1, R2, R3, and R4 are variables in Eq. (4). The relative errors can be determined from Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration http://dx.doi.org/10.5772/intechopen.79598 157

$$\mathbf{X} = \frac{1}{4} \begin{bmatrix} -\mathbf{R}\_1 \boldsymbol{\Theta}\_1 & \mathbf{R}\_2 \boldsymbol{\Theta}\_2 & -\mathbf{R}\_3 \boldsymbol{\Theta}\_3 & \mathbf{R}\_4 \boldsymbol{\Theta}\_4 \\ \mathbf{R}\_1 \boldsymbol{\Theta}\_1 & \mathbf{R}\_2 \boldsymbol{\Theta}\_2 & \mathbf{R}\_3 \boldsymbol{\Theta}\_3 & \mathbf{R}\_4 \boldsymbol{\Theta}\_4 \\ \frac{\mathbf{R}\_1}{\mathbf{I}\_1 + \mathbf{L}} \boldsymbol{\Theta}\_1 & -\frac{\mathbf{R}\_2}{\mathbf{I}\_2 + \mathbf{L}} \boldsymbol{\Theta}\_2 & -\frac{\mathbf{R}\_3}{\mathbf{I}\_3 + \mathbf{L}} \boldsymbol{\Theta}\_3 & \frac{\mathbf{R}\_4}{\mathbf{I}\_4 + \mathbf{L}} \boldsymbol{\Theta}\_4 \end{bmatrix} \tag{5}$$

leading to

3.2. Error modeling

Figure 6. Motion model of the mobile platform.

Figure 5. Roller deformation.

156 Applications of Mobile Robots

where <sup>X</sup> <sup>¼</sup> <sup>V</sup>t is X½ � ;Y; <sup>θ</sup> <sup>T</sup>

By multiplying time t, Eq. (3) becomes a displacement equation as

. θ ¼ Wt is ½ � θ1; θ2; θ3; θ<sup>4</sup>

T

l1, l2, l3, l4, R1, R2, R3, and R4 are variables in Eq. (4). The relative errors can be determined from

X ¼ D1θ (4)

. Individual geometric parameters including

$$
\Delta \mathbf{X} = \mathbf{G} \Delta \mathbf{B} \tag{6}
$$

where <sup>Δ</sup><sup>X</sup> <sup>¼</sup> ½ � <sup>Δ</sup>X; <sup>Δ</sup>Y; <sup>Δ</sup><sup>θ</sup> <sup>T</sup> , ΔB ¼ ½ � ΔR1; ΔR2; ΔR3; ΔR4; Δl1; Δl2; Δl3; Δl4 T , and

G ¼ �θ<sup>1</sup> θ<sup>2</sup> �θ<sup>3</sup> θ<sup>1</sup> θ<sup>2</sup> θ<sup>3</sup> θ1 M1 � <sup>θ</sup><sup>2</sup> M2 � <sup>θ</sup><sup>3</sup> M3 θ<sup>4</sup> 0 0 θ<sup>4</sup> 0 0 θ4 M4 � R1θ<sup>1</sup> M1 2 R2θ<sup>2</sup> M2 2 0 0 0 0 R3θ<sup>3</sup> M3 <sup>2</sup> � R4θ<sup>4</sup> M4 2 2 6 6 6 6 4 3 7 7 7 7 5 R1 ¼ R0 þ ΔR1, l1 ¼ l0 þ Δl1, R2 ¼ R0 þ ΔR2, l2 ¼ l0 þ Δl2 R3 ¼ R0 þ ΔR3, l3 ¼ l0 þ Δl3, R4 ¼ R0 þ ΔR4, l4 ¼ l0 þ Δl4

With the least squares method, the geometric errors can be solved as

$$
\Delta \mathbf{B} = \left(\mathbf{G}^{\mathrm{T}} \mathbf{G}\right)^{-1} \mathbf{G}^{\mathrm{T}} \Delta \mathbf{X} \tag{7}
$$

ΔRi is generally defined as the tolerance resulting from manufacturing or assembly. With the deformation of the roller and Δli limited to ½ � –h; h and ½ � –j; j ðh ¼ 3mm, j ¼ 5mm) respectively, li and Ri can be defined as:

$$\mathbf{l}\_{i} = 2\mathbf{j}\mathbf{r} + (\mathbf{l}\_{0} - \mathbf{j})\tag{8}$$

$$\mathbf{R}\_{i} = 2\mathbf{h}\mathbf{r} + (\mathbf{R}\_{0} - \mathbf{h})\tag{9}$$

where li and Ri are in l½ � <sup>0</sup> � j; l0 þ j and R½ � <sup>0</sup> � h;R0 þ h , respectively, and r is a random number between 0 and 1. The angular speeds of all wheels are set to 20rad=s, and the time t is set to 1 s.

The rank of matrix G in Eq. (7) is generally 3, so G is a full rank matrix and ΔB can be obtained. The following steps can be carried out. First, the displacement error measurement is analyzed. In order to obtain ΔB, it is needed to obtain the displacement error matrix ΔX first.

The system is moved in the YG direction and stopped every 2 s to obtain its position and posture <sup>G</sup>Xk ; <sup>G</sup>Yk ; <sup>G</sup>θ<sup>k</sup> � �. The principle of measurement is shown in Figure 7 and ΔX can be calculated as

$$\mathbf{dX} = \begin{bmatrix} \left( ^{\mathcal{G}}\mathbf{X}\_{1} + ^{\mathcal{G}}\mathbf{X}\_{2} + ^{\mathcal{G}}\mathbf{X}\_{3} + ^{\mathcal{G}}\mathbf{X}\_{4} \right) / 10 - \mathbf{X}\_{\mathcal{T}}\\ \left( ^{\mathcal{G}}\mathbf{Y}\_{1} + ^{\mathcal{G}}\mathbf{Y}\_{2} + ^{\mathcal{G}}\mathbf{Y}\_{3} + ^{\mathcal{G}}\mathbf{Y}\_{4} \right) / 10 - \mathbf{Y}\_{\mathcal{T}}\\ \left( ^{\mathcal{G}}\boldsymbol{\Theta}\_{1} + ^{\mathcal{G}}\boldsymbol{\Theta}\_{2} + ^{\mathcal{G}}\boldsymbol{\Theta}\_{3} + ^{\mathcal{G}}\boldsymbol{\Theta}\_{4} \right) / 10 - \boldsymbol{\Theta}\_{\mathcal{T}} \end{bmatrix} \tag{10}$$

Figure 7. The principle of measuring displacement.

According to Eq. (2), XT, YT, and θ<sup>T</sup> are theoretical values which can be obtained as follows: XT ¼ Vxt, YT ¼ Vyt, θ<sup>T</sup> ¼ 0. Finally, through the experiment, the displacement errors can be obtained.

$$
\Delta \mathbf{X} = \begin{bmatrix} \text{3mm} \\ \text{1mm} \\ \mathbf{0}^\circ \end{bmatrix} \tag{11}
$$

Figure 8. The values obtained with the Monte Carlo method.

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

http://dx.doi.org/10.5772/intechopen.79598

159

Substituting Eq. (11) in to Eq. (7), ΔB can be determined.

The Monte Carlo analysis is applied to deal with 50 samples. While ΔX has been given in Eq. (11), the corresponding ΔB should also satisfy its own tolerance, i.e., �3 ≤ ΔRi ≤ 3 and �5 ≤ ΔLi ≤ 5.

The results are given in Figure 8. These data are averaged to form a new result: ΔB ¼ ½ � �0:541; <sup>1</sup>:237; �0:605; <sup>1</sup>:055; <sup>0</sup>:458; <sup>0</sup>:683; �0:048; �0:<sup>683</sup> <sup>T</sup> : A process capability index is then used to estimate the value of ΔB. Process capability refers to the ability of a process to meet the required quality. It is a measure of the minimum fluctuation of the internal consistency of the process itself in the most stable state. When the process is stable, the quality characteristic value of the product is in the range ½ � μ � 3σ; μ þ 3σ , where μ is the ensemble average of the quality characteristic value of the product, and σ is the standard deviation of the quality characteristic value of the product.

There are two kinds of situations for which the process capability index has to be solved. First, μ ¼ M as shown in Figure 9: USL is the maximum error of quality and Lsl is the minimum error of quality; M ¼ ð Þ USL þ LSL =2; μ is the average value of process machining; and Cp indicates the process capability index, and Cp ¼ ð Þ USL � LSL =6σ:

Second, μ 6¼ M as shown in Figure 10: Cpk indicates the process capability index, and Cpk ¼ ð Þ USL � LSL =6σ � ∣M � μ∣=3σ. Only process capability index greater than 1 is valid. As �3 ≤ ΔR<sup>i</sup> ≤ 3 and �5 ≤ ΔL<sup>i</sup> ≤ 5, M ¼ 0 and μ 6¼ M, the following results can be obtained: Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration http://dx.doi.org/10.5772/intechopen.79598 159

Figure 8. The values obtained with the Monte Carlo method.

According to Eq. (2), XT, YT, and θ<sup>T</sup> are theoretical values which can be obtained as follows: XT ¼ Vxt, YT ¼ Vyt, θ<sup>T</sup> ¼ 0. Finally, through the experiment, the displacement errors can be

> 2 6 4

The Monte Carlo analysis is applied to deal with 50 samples. While ΔX has been given in Eq. (11), the corresponding ΔB should also satisfy its own tolerance, i.e., �3 ≤ ΔRi ≤ 3 and

The results are given in Figure 8. These data are averaged to form a new result: ΔB ¼ ½ � �0:541; <sup>1</sup>:237; �0:605; <sup>1</sup>:055; <sup>0</sup>:458; <sup>0</sup>:683; �0:048; �0:<sup>683</sup> <sup>T</sup>: A process capability index is then used to estimate the value of ΔB. Process capability refers to the ability of a process to meet the required quality. It is a measure of the minimum fluctuation of the internal consistency of the process itself in the most stable state. When the process is stable, the quality characteristic value of the product is in the range ½ � μ � 3σ; μ þ 3σ , where μ is the ensemble average of the quality characteristic value of the product, and σ is the standard deviation of the quality

There are two kinds of situations for which the process capability index has to be solved. First, μ ¼ M as shown in Figure 9: USL is the maximum error of quality and Lsl is the minimum error of quality; M ¼ ð Þ USL þ LSL =2; μ is the average value of process machining; and Cp indicates

Second, μ 6¼ M as shown in Figure 10: Cpk indicates the process capability index, and Cpk ¼ ð Þ USL � LSL =6σ � ∣M � μ∣=3σ. Only process capability index greater than 1 is valid. As �3 ≤ ΔR<sup>i</sup> ≤ 3 and �5 ≤ ΔL<sup>i</sup> ≤ 5, M ¼ 0 and μ 6¼ M, the following results can be obtained:

3mm 1mm 0 �

3 7

<sup>5</sup> (11)

ΔX ¼

Substituting Eq. (11) in to Eq. (7), ΔB can be determined.

Figure 7. The principle of measuring displacement.

the process capability index, and Cp ¼ ð Þ USL � LSL =6σ:

obtained.

158 Applications of Mobile Robots

�5 ≤ ΔLi ≤ 5.

characteristic value of the product.

Cpkð Þ¼ ΔR<sup>1</sup> 1:58, Cpkð Þ¼ Δl<sup>1</sup> 0:57, Cpkð Þ¼ ΔR<sup>2</sup> 1:06, Cpkð Þ¼ Δl<sup>2</sup> 0:58, Cpkð Þ¼ ΔR<sup>3</sup> 1:54, Cpkð Þ¼ Δl<sup>3</sup> 0:63, Cpkð Þ¼ ΔR<sup>4</sup> 1:17, and Cpkð Þ¼ Δl<sup>4</sup> 0:5.

to 1 mm; there are 7 values of <sup>Δ</sup>Ri, and 11 values of <sup>Δ</sup>li, so there are 7<sup>4</sup> � 114 groups of combinations of matrix G: Take all the combinations of matrix G to Eq. (6) to obtain ΔB and exclude the cases for which ΔR<sup>i</sup> and Δl<sup>i</sup> are not in ½ � �3; 3 and ½ � �5; 5 , respectively. Thus, 277 groups of ΔB can be obtained. As shown in Figure 11, the average values of ΔR1, ΔR2, ΔR3, ΔR4, Δl1, Δl2, Δl3, and

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

, which is

161

http://dx.doi.org/10.5772/intechopen.79598

<sup>Δ</sup>l4, lead to a new <sup>Δ</sup>B2 ¼ �½ � <sup>0</sup>:233; <sup>1</sup>:467; �0:932; <sup>0</sup>:824; <sup>0</sup>:2; �0:213; �0:068; �0:<sup>039</sup> <sup>T</sup>

close to ΔB1.

Figure 11. The values obtained in interval analysis.

$$\text{Figure 9. } \mu = \text{M}$$

Figure 10. μ 6¼ M.

The values of ΔR1, ΔR2, ΔR3, and ΔR4 are closer to the real values than the values of Δl1, <sup>Δ</sup>l2, <sup>Δ</sup>l3, and <sup>Δ</sup>l4. Now, taking <sup>Δ</sup>B1ð Þ¼ � <sup>a</sup> ½ � <sup>0</sup>:541; <sup>1</sup>:237; �0:605; <sup>1</sup>:055; <sup>0</sup>; <sup>0</sup>; <sup>0</sup>; <sup>0</sup> <sup>T</sup> , ΔB1ð Þ¼ b ½ � <sup>0</sup>; <sup>0</sup>; <sup>0</sup>; <sup>0</sup>; <sup>0</sup>:458; <sup>0</sup>:683; �0:048; �0:<sup>683</sup> <sup>T</sup> in Eq. (5), it can be seen that <sup>Δ</sup>X1 ¼ �½ <sup>0</sup>:4119,4; �0:0003� T , <sup>Δ</sup>X1ð Þ¼ <sup>b</sup> ½ � <sup>0</sup>; <sup>0</sup>; �0:<sup>0003</sup> <sup>T</sup> . According to the change rate relative to Eq. (10), the influence of ΔR1, ΔR2, ΔR3, and ΔR4 is much bigger than that of Δl1, Δl2, Δl3, and Δl4. Although the values of Δl1, Δl2, Δl3, and Δl4 are not accurate enough for the real values, ΔB1 is valid.

The interval analysis is also carried out. In Eq. (6), the value of G is uncertain because of the change of ΔR<sup>i</sup> and Δl<sup>i</sup> in ½ � �3; 3 and ½ � �5; 5 , respectively. Now, the increment of ΔR<sup>i</sup> and Δl<sup>i</sup> is set to 1 mm; there are 7 values of <sup>Δ</sup>Ri, and 11 values of <sup>Δ</sup>li, so there are 7<sup>4</sup> � 114 groups of combinations of matrix G: Take all the combinations of matrix G to Eq. (6) to obtain ΔB and exclude the cases for which ΔR<sup>i</sup> and Δl<sup>i</sup> are not in ½ � �3; 3 and ½ � �5; 5 , respectively. Thus, 277 groups of ΔB can be obtained. As shown in Figure 11, the average values of ΔR1, ΔR2, ΔR3, ΔR4, Δl1, Δl2, Δl3, and <sup>Δ</sup>l4, lead to a new <sup>Δ</sup>B2 ¼ �½ � <sup>0</sup>:233; <sup>1</sup>:467; �0:932; <sup>0</sup>:824; <sup>0</sup>:2; �0:213; �0:068; �0:<sup>039</sup> <sup>T</sup> , which is close to ΔB1.

Figure 11. The values obtained in interval analysis.

Cpkð Þ¼ ΔR<sup>1</sup> 1:58, Cpkð Þ¼ Δl<sup>1</sup> 0:57, Cpkð Þ¼ ΔR<sup>2</sup> 1:06, Cpkð Þ¼ Δl<sup>2</sup> 0:58, Cpkð Þ¼ ΔR<sup>3</sup> 1:54,

The values of ΔR1, ΔR2, ΔR3, and ΔR4 are closer to the real values than the values of Δl1,

½ � <sup>0</sup>; <sup>0</sup>; <sup>0</sup>; <sup>0</sup>; <sup>0</sup>:458; <sup>0</sup>:683; �0:048; �0:<sup>683</sup> <sup>T</sup> in Eq. (5), it can be seen that <sup>Δ</sup>X1 ¼ �½ <sup>0</sup>:4119,4;

influence of ΔR1, ΔR2, ΔR3, and ΔR4 is much bigger than that of Δl1, Δl2, Δl3, and Δl4. Although the values of Δl1, Δl2, Δl3, and Δl4 are not accurate enough for the real values, ΔB1 is valid.

The interval analysis is also carried out. In Eq. (6), the value of G is uncertain because of the change of ΔR<sup>i</sup> and Δl<sup>i</sup> in ½ � �3; 3 and ½ � �5; 5 , respectively. Now, the increment of ΔR<sup>i</sup> and Δl<sup>i</sup> is set

, <sup>Δ</sup>X1ð Þ¼ <sup>b</sup> ½ � <sup>0</sup>; <sup>0</sup>; �0:<sup>0003</sup> T. According to the change rate relative to Eq. (10), the

, ΔB1ð Þ¼ b

<sup>Δ</sup>l2, <sup>Δ</sup>l3, and <sup>Δ</sup>l4. Now, taking <sup>Δ</sup>B1ð Þ¼ � <sup>a</sup> ½ � <sup>0</sup>:541; <sup>1</sup>:237; �0:605; <sup>1</sup>:055; <sup>0</sup>; <sup>0</sup>; <sup>0</sup>; <sup>0</sup> <sup>T</sup>

�0:0003�

T

Figure 10. μ 6¼ M.

Figure 9. μ ¼ M.

160 Applications of Mobile Robots

Cpkð Þ¼ Δl<sup>3</sup> 0:63, Cpkð Þ¼ ΔR<sup>4</sup> 1:17, and Cpkð Þ¼ Δl<sup>4</sup> 0:5.

The same method is used to solve the process capability index of ΔB2. One obtains: Cpkð Þ¼ ΔR<sup>1</sup> 3:34, Cpkð Þ¼ Δl<sup>1</sup> 0:68, Cpkð Þ¼ ΔR<sup>2</sup> 1:22, Cpkð Þ¼ Δl<sup>2</sup> 0:59, Cpkð Þ¼ ΔR<sup>3</sup> 2:5, Cpk ð Þ¼ Δl<sup>3</sup> 0:63, Cpkð Þ¼ R4 1:73, and Cpkð Þ¼ Δl4 0:6. The values of ΔR1, ΔR2, ΔR3, and ΔR4 in ΔB2 are closer to the real ones than the values of Δl1, Δl2, Δl3, and Δl4 in ΔB2. As the influence of ΔR1, ΔR2, ΔR3, and ΔR4 is bigger than that of Δl1, Δl2, Δl3, and Δl4, ΔB2 is valid.

and S<sup>2</sup> ¼ ð Þ D<sup>2</sup> � D<sup>0</sup> W2t. The results of the two correction methods are almost identical in

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

http://dx.doi.org/10.5772/intechopen.79598

163

The experiment with four movements is shown in Figure 12(a), while the actual movement is shown in Figure 12(b). It can be found that S<sup>1</sup> and S<sup>2</sup> are close to the measured displacement

The localization component of mobile manipulator includes two laser range finders and a number of reflectors that are of cylindrical shape placed in the environment. Each reflector is

To achieve accurate localization, the system should have the ability to perceive external information through extracting the reflector features. In this research, as shown in Figure 13, there are n reflectors, and the feature of each reflector Bið Þ i ¼ 1; 2;…; n is extracted from the raw data. The feature extraction algorithm consists of three steps: (i) filtering and clustering, (ii)

The first step is filtering and clustering. The raw data obtained by each finder are a set of discrete data sequences f g ð Þ <sup>γ</sup>; <sup>∅</sup> ; <sup>λ</sup> <sup>i</sup> <sup>j</sup> <sup>i</sup> <sup>¼</sup> <sup>1</sup>; <sup>2</sup>;…; <sup>n</sup> . <sup>γ</sup> is the distance from the target point to the finder. ∅ is the polar angle. λ<sup>i</sup> is the intensity value of the ith data point. In the process of

The density of the collected data points is proportional to the distance from the target point to the laser range finder. To improve the efficiency of the feature extraction process, an adaptive clustering method is adopted as given by Eq. (12). Unless the distance between two data points

data analysis, the outlier points that are contaminated by noise will be filtered out.

is less than the threshold δ, these data points are clustered for one reflector.

errors, so both ΔB1 and ΔB2 are satisfied to revise the matrix D1.

covered by a reflective tape with high reflectivity (Figure 13).

theory.

4. Localization

4.1. System configuration

4.2. Dynamic tracking

identification, and (iii) feature extraction.

Figure 13. Localization system for the mobile manipulator.

The result is verified as well. ΔB1 is used to revise the parameters of D1 in Eq. (2) as

$$\mathbf{D}\_{1} = \frac{1}{4} \begin{bmatrix} -186.959 & 188.737 & -186.895 & 188.555 \\ 186.959 & 188.737 & 186.895 & 188.555 \\ 0.1369 & -0.1382 & -0.1369 & 0.1382 \end{bmatrix}$$

By setting two sets of four wheel speeds <sup>W</sup><sup>1</sup> <sup>¼</sup> ½ � <sup>10</sup>π=21; <sup>10</sup>π=21; <sup>10</sup>π=21; <sup>10</sup>π=<sup>21</sup> <sup>T</sup> and <sup>W</sup><sup>2</sup> <sup>¼</sup> ½ � <sup>4</sup>π=7; <sup>4</sup>π=7; <sup>4</sup>π=7; <sup>4</sup>π=<sup>7</sup> <sup>T</sup> , the displacement errors can be computed as S<sup>1</sup> ¼ ð Þ D<sup>1</sup> � D<sup>0</sup> W2t

Figure 12. (a) Schematic diagram of measuring and (b) the actual measuring.

and S<sup>2</sup> ¼ ð Þ D<sup>2</sup> � D<sup>0</sup> W2t. The results of the two correction methods are almost identical in theory.

The experiment with four movements is shown in Figure 12(a), while the actual movement is shown in Figure 12(b). It can be found that S<sup>1</sup> and S<sup>2</sup> are close to the measured displacement errors, so both ΔB1 and ΔB2 are satisfied to revise the matrix D1.

#### 4. Localization

The same method is used to solve the process capability index of ΔB2. One obtains: Cpkð Þ¼ ΔR<sup>1</sup> 3:34, Cpkð Þ¼ Δl<sup>1</sup> 0:68, Cpkð Þ¼ ΔR<sup>2</sup> 1:22, Cpkð Þ¼ Δl<sup>2</sup> 0:59, Cpkð Þ¼ ΔR<sup>3</sup> 2:5, Cpk ð Þ¼ Δl<sup>3</sup> 0:63, Cpkð Þ¼ R4 1:73, and Cpkð Þ¼ Δl4 0:6. The values of ΔR1, ΔR2, ΔR3, and ΔR4 in ΔB2 are closer to the real ones than the values of Δl1, Δl2, Δl3, and Δl4 in ΔB2. As the influence

> �186:959 188:737 �186:895 186:959 188:737 186:895 0:1369 �0:1382 �0:1369

By setting two sets of four wheel speeds <sup>W</sup><sup>1</sup> <sup>¼</sup> ½ � <sup>10</sup>π=21; <sup>10</sup>π=21; <sup>10</sup>π=21; <sup>10</sup>π=<sup>21</sup> <sup>T</sup> and <sup>W</sup><sup>2</sup> <sup>¼</sup>

188:555 188:555 0:1382

, the displacement errors can be computed as S<sup>1</sup> ¼ ð Þ D<sup>1</sup> � D<sup>0</sup> W2t

of ΔR1, ΔR2, ΔR3, and ΔR4 is bigger than that of Δl1, Δl2, Δl3, and Δl4, ΔB2 is valid. The result is verified as well. ΔB1 is used to revise the parameters of D1 in Eq. (2) as

> <sup>D</sup><sup>1</sup> <sup>¼</sup> <sup>1</sup> 4

½ � <sup>4</sup>π=7; <sup>4</sup>π=7; <sup>4</sup>π=7; <sup>4</sup>π=<sup>7</sup> <sup>T</sup>

162 Applications of Mobile Robots

Figure 12. (a) Schematic diagram of measuring and (b) the actual measuring.

#### 4.1. System configuration

The localization component of mobile manipulator includes two laser range finders and a number of reflectors that are of cylindrical shape placed in the environment. Each reflector is covered by a reflective tape with high reflectivity (Figure 13).

#### 4.2. Dynamic tracking

To achieve accurate localization, the system should have the ability to perceive external information through extracting the reflector features. In this research, as shown in Figure 13, there are n reflectors, and the feature of each reflector Bið Þ i ¼ 1; 2;…; n is extracted from the raw data. The feature extraction algorithm consists of three steps: (i) filtering and clustering, (ii) identification, and (iii) feature extraction.

The first step is filtering and clustering. The raw data obtained by each finder are a set of discrete data sequences f g ð Þ <sup>γ</sup>; <sup>∅</sup> ; <sup>λ</sup> <sup>i</sup> <sup>j</sup> <sup>i</sup> <sup>¼</sup> <sup>1</sup>; <sup>2</sup>;…; <sup>n</sup> . <sup>γ</sup> is the distance from the target point to the finder. ∅ is the polar angle. λ<sup>i</sup> is the intensity value of the ith data point. In the process of data analysis, the outlier points that are contaminated by noise will be filtered out.

The density of the collected data points is proportional to the distance from the target point to the laser range finder. To improve the efficiency of the feature extraction process, an adaptive clustering method is adopted as given by Eq. (12). Unless the distance between two data points is less than the threshold δ, these data points are clustered for one reflector.

Figure 13. Localization system for the mobile manipulator.

$$\delta = \gamma\_{i-1} \left( (\sin \Delta \mathfrak{Q}) / \left( \sin \left( \mathfrak{F} - \Delta \mathfrak{Q} \right) \right) \right) + 3\sigma\_{\gamma} \tag{12}$$

where <sup>γ</sup><sup>i</sup>�<sup>1</sup> is the distance value of the ið Þ � <sup>1</sup> th data point, <sup>Δ</sup><sup>∅</sup> is the angle resolution of the laser range finder, β is an auxiliary constant parameter, and σγ is the measurement error. The values of parameters σγ and β are given as 0.01 m and 10� , respectively.

The second step is identification. After clustering, the data set can be correlated to each observable reflector. Each reflector data set is then used to calculate the position of the reflector in the laser range finder frame [19]. Let λδ be the reflected intensity threshold and D½ � � Dδ; D þ D<sup>δ</sup> be the diameter range of a reflector, where D is a nominal reflector diameter and D<sup>δ</sup> is the tolerance. The values of λδ and D<sup>δ</sup> are selected based on the actual situation. Considering that Wc represents a set after clustering, i.e., W<sup>c</sup> ¼ f g ð Þ γ; ∅ ; λ <sup>i</sup> <sup>j</sup><sup>i</sup> <sup>¼</sup> � m; …; ng, for each reflector, the measured diameter Dc is calculated as Dc <sup>¼</sup> ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi γ2 <sup>n</sup> þ γ<sup>2</sup> <sup>m</sup> � 2γnγ<sup>m</sup> cos ð Þ ∅<sup>n</sup> � ∅<sup>m</sup> <sup>p</sup> , where ð Þ <sup>γ</sup>; <sup>∅</sup> <sup>m</sup> and ð Þ <sup>γ</sup>; <sup>∅</sup> <sup>n</sup> are the beginning and end data of a reflector set, respectively. When the set Wc satisfies the following two conditions

$$\begin{cases} \lambda\_{\text{max}} \ge \lambda\_{\text{\ $}} \\ \mathbf{D}\_{\text{\$ }} \in [\mathbf{D}\_{\text{min}}, \mathbf{D}\_{\text{max}}] \end{cases} \tag{13}$$

Furthermore, as mentioned before, the position of the finder in the platform frame is

The optimal triangulation localization algorithm based on angle measurement is carried out. A number of experiments were carried out on the reflector feature extraction algorithm before proposing a localization algorithm for the system. A single reflector was placed in the measuring range of the finder. The finder performed repeated scanning when it was placed at difference places. Two different feature extraction algorithms were tested to calculate the position of the reflector, and the results are shown in Figure 15. Algorithm A is the feature extraction algorithm proposed in this chapter, while algorithm B is the least square circle fitting method without a radius constraint. Apparently, the former one yields a better result.

Px L2 <sup>þ</sup> <sup>P</sup>γ<sup>B</sup> cos <sup>P</sup>β<sup>B</sup> � �<sup>2</sup>

After extracting the reflector center, the positions of all the reflectors G <sup>¼</sup> <sup>L</sup>γ<sup>B</sup>

max � <sup>L</sup>β<sup>B</sup> � � min � �π<sup>∙</sup>

<sup>R</sup><sup>γ</sup> <sup>¼</sup> <sup>L</sup>γ<sup>B</sup> � �

1; 2; …; ng can be obtained and used to calculate the distance measurement range Rγ and the

!

max <sup>∈</sup> G, <sup>L</sup>β<sup>B</sup>

max � <sup>L</sup>γ<sup>B</sup> � �

� �

1 n Xn i¼1 L γB

min ∈ G.

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

<sup>P</sup>yL2 � <sup>P</sup><sup>γ</sup> <sup>B</sup> sin <sup>P</sup>β<sup>B</sup>Þ<sup>=</sup> PxL2 � <sup>P</sup>γ<sup>B</sup> cos <sup>P</sup>β<sup>B</sup>

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

<sup>þ</sup> PyL2 <sup>þ</sup> <sup>P</sup>γ<sup>B</sup> sin <sup>P</sup>β<sup>B</sup> � �<sup>2</sup>

� �

� � in the platform frame can be

http://dx.doi.org/10.5772/intechopen.79598

(18)

165

<sup>L</sup>β<sup>B</sup> � �

n

=180 (20)

min (19)

i ji ¼

<sup>P</sup><sup>x</sup> L2; <sup>P</sup><sup>y</sup> L2 � �. The position of reflector center B <sup>P</sup><sup>γ</sup> <sup>B</sup>; <sup>P</sup><sup>β</sup> <sup>B</sup>

q

<sup>P</sup>β<sup>B</sup> <sup>¼</sup> arctan<sup>ð</sup>

angler measurement range R<sup>β</sup> of the reflector, as given below:

<sup>R</sup><sup>β</sup> <sup>¼</sup> <sup>L</sup>β<sup>B</sup>

max <sup>∈</sup> G, <sup>L</sup>γ<sup>B</sup>

� �

� �

min <sup>∈</sup> G, <sup>L</sup>β<sup>B</sup>

� �

<sup>P</sup>γ<sup>B</sup> <sup>¼</sup>

Figure 14. Extraction of the center of the cylindrical reflector.

8 < :

expressed as

where <sup>L</sup>γ<sup>B</sup> � �

then the set W<sup>c</sup> is established for all reflectors.

The third step is feature extraction. The feature extraction algorithm of a reflector extracts its central position γL,B; βL,<sup>B</sup> � � in the laser range finder frame Xf g <sup>L</sup>; YL;ZL , where <sup>γ</sup>L,<sup>B</sup> and <sup>β</sup>L,<sup>B</sup> are the distance and azimuth of each reflector, respectively. In the process of extracting the feature of the circular reflector, only a small portion of the entire circle was scanned with noise, so the accuracy of the fitting result would be low if a general least square circle fitting method is used. Since the radius of the reflector is known, the known radius is used as a constraint to improve the accuracy.

First, the value of βL,<sup>B</sup> is obtained from nð Þ � m consecutive data points of the reflector set

$$\prescript{\rm L}{}{\beta}\_B = \frac{1}{(\mathbf{n} - \mathbf{m})} \sum\_{i=\mathbf{m}}^n \prescript{\rm L}{}{\beta}\_i \tag{14}$$

As shown in Figure 14, Mi represents the ith data and OL2B is a line between the reflector center B and the laser range finder center OL2; the projected angle of the line OL2B in the laser range finder frame is <sup>L</sup>βB, while the angle between the line OL2B and the line OL2Mi is θi. The distance from Mi to B is approximately the radius of the reflector, i.e., BMi � � � � ¼ D=2:

$$
\partial\_{\mathbf{i}} = \mathcal{Q}\mathbf{j} - \prescript{\mathbf{L}}{}{\mathcal{B}}\_{\mathcal{B}} \tag{15}
$$

$$\left| \overline{\mathbf{O}\_{\mathsf{L}2}} \mathbf{B} \right|\_{\mathrm{i}} = \left| \overline{\mathbf{O}\_{\mathsf{L}2} \mathbf{M}\_{\mathrm{i}}} \right| \cos \left| \theta\_{\mathrm{i}} \right| + \left| \overline{\mathbf{B} \mathbf{M}\_{\mathrm{i}}} \right| \cos \left( \arcsin \left( \left( 2 \cdot \left| \overline{\mathbf{O}\_{\mathsf{L}2} \mathbf{M}\_{\mathrm{i}}} \right| \sin \theta\_{\mathrm{i}} \right) / \mathbf{D} \right) \right) \tag{16}$$

$$\mathbf{^L \gamma}\_B = \frac{1}{\mathbf{n} - \mathbf{m}} \sum\_{i=\mathbf{m}}^n \left| \overline{\mathbf{O\_{L2}}} \right|\_{\mathbf{i}'} \quad \mathbf{i} = 1, 2, \cdots \mathbf{m} \tag{17}$$

Figure 14. Extraction of the center of the cylindrical reflector.

<sup>δ</sup> <sup>¼</sup> <sup>γ</sup><sup>i</sup>�<sup>1</sup> ð Þ sin <sup>Δ</sup><sup>∅</sup> <sup>=</sup> sin <sup>β</sup> � <sup>Δ</sup><sup>∅</sup> � � � � � � <sup>þ</sup> <sup>3</sup>σγ (12)

, respectively.

<sup>j</sup><sup>i</sup> <sup>¼</sup> �

(13)

where <sup>γ</sup><sup>i</sup>�<sup>1</sup> is the distance value of the ið Þ � <sup>1</sup> th data point, <sup>Δ</sup><sup>∅</sup> is the angle resolution of the laser range finder, β is an auxiliary constant parameter, and σγ is the measurement error. The

The second step is identification. After clustering, the data set can be correlated to each observable reflector. Each reflector data set is then used to calculate the position of the reflector in the laser range finder frame [19]. Let λδ be the reflected intensity threshold and D½ � � Dδ; D þ D<sup>δ</sup> be the diameter range of a reflector, where D is a nominal reflector diameter and D<sup>δ</sup> is the tolerance. The values of λδ and D<sup>δ</sup> are selected based on the actual situation. Considering that Wc represents a set after clustering, i.e., W<sup>c</sup> ¼ f g ð Þ γ; ∅ ; λ <sup>i</sup>

m; …; ng, for each reflector, the measured diameter Dc is calculated as

Dc ∈ ½ � Dmin; Dmax

The third step is feature extraction. The feature extraction algorithm of a reflector extracts its

are the distance and azimuth of each reflector, respectively. In the process of extracting the feature of the circular reflector, only a small portion of the entire circle was scanned with noise, so the accuracy of the fitting result would be low if a general least square circle fitting method is used. Since the radius of the reflector is known, the known radius is used as a constraint to

First, the value of βL,<sup>B</sup> is obtained from nð Þ � m consecutive data points of the reflector set

ð Þ n � m

As shown in Figure 14, Mi represents the ith data and OL2B is a line between the reflector center B and the laser range finder center OL2; the projected angle of the line OL2B in the laser range finder frame is <sup>L</sup>βB, while the angle between the line OL2B and the line OL2Mi is θi. The

Xn i¼m

<sup>L</sup>β<sup>B</sup> <sup>¼</sup> <sup>1</sup>

distance from Mi to B is approximately the radius of the reflector, i.e., BMi

� cos θ<sup>i</sup> j j þ BMi � � �

> Xn i¼m

OL2B � � � � i

n � m

<sup>L</sup>γ<sup>B</sup> <sup>¼</sup> <sup>1</sup>

data of a reflector set, respectively. When the set Wc satisfies the following two conditions

(

<sup>p</sup> , where ð Þ <sup>γ</sup>; <sup>∅</sup> <sup>m</sup> and ð Þ <sup>γ</sup>; <sup>∅</sup> <sup>n</sup> are the beginning and end

λmax ≥ λδ

in the laser range finder frame Xf g <sup>L</sup>; YL;ZL , where γL,<sup>B</sup> and βL,<sup>B</sup>

<sup>L</sup>β<sup>i</sup> (14)

� ¼ D=2:

� � �

� sin θ<sup>i</sup> � �=D � � � � (16)

, i ¼ 1, 2, ⋯m (17)

<sup>θ</sup><sup>i</sup> <sup>¼</sup> <sup>∅</sup><sup>i</sup> � <sup>L</sup>β<sup>B</sup> (15)

� cos arcsin 2∙ OL2Mi � � �

values of parameters σγ and β are given as 0.01 m and 10�

Dc <sup>¼</sup> ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

then the set W<sup>c</sup> is established for all reflectors.

� �

<sup>m</sup> � 2γnγ<sup>m</sup> cos ð Þ ∅<sup>n</sup> � ∅<sup>m</sup>

γ2 <sup>n</sup> þ γ<sup>2</sup>

164 Applications of Mobile Robots

central position γL,B; βL,<sup>B</sup>

improve the accuracy.

OL2B � � � �

<sup>i</sup> ¼ OL2Mi � � � Furthermore, as mentioned before, the position of the finder in the platform frame is <sup>P</sup><sup>x</sup> L2; <sup>P</sup><sup>y</sup> L2 � �. The position of reflector center B <sup>P</sup><sup>γ</sup> <sup>B</sup>; <sup>P</sup><sup>β</sup> <sup>B</sup> � � in the platform frame can be expressed as

$$\begin{cases} \prescript{\text{P}}{}{\text{\text{\textdegree}}}{\text{\textdegree}} \mathbf{y}\_{\text{B}} = \sqrt{\left(\prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}} + \prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}} \mathbf{y}\_{\text{B}}\right)^{2} + \left(\prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}} + \prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}} \sin^{\text{P}}{\text{\textdegree}}{}{\text{\textdegree}}\right)^{2} \\ \prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}} = \arctan(\prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}} - \prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}} \sin^{\text{P}}{\text{\textdegree}}{}{\text{\textdegree}}/\left(\prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}}\_{\text{B}}/\left(\prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}}\_{\text{B}} - \prescript{\text{P}}{}{\text{\textdegree}}{}{\text{\textdegree}}{\text{\textdegree}}{\text{\textdegree}}{\text{\textdegree}}\right) \end{cases} \tag{18}$$

The optimal triangulation localization algorithm based on angle measurement is carried out. A number of experiments were carried out on the reflector feature extraction algorithm before proposing a localization algorithm for the system. A single reflector was placed in the measuring range of the finder. The finder performed repeated scanning when it was placed at difference places. Two different feature extraction algorithms were tested to calculate the position of the reflector, and the results are shown in Figure 15. Algorithm A is the feature extraction algorithm proposed in this chapter, while algorithm B is the least square circle fitting method without a radius constraint. Apparently, the former one yields a better result.

After extracting the reflector center, the positions of all the reflectors G <sup>¼</sup> <sup>L</sup>γ<sup>B</sup> <sup>L</sup>β<sup>B</sup> � � i ji ¼ n 1; 2; …; ng can be obtained and used to calculate the distance measurement range Rγ and the angler measurement range R<sup>β</sup> of the reflector, as given below:

$$\mathbf{R}\_{\mathbf{\dot{\gamma}}} = \left( {}^{\mathcal{L}}\boldsymbol{\gamma}\_{\mathcal{B}} \right)\_{\text{max}} - \left( {}^{\mathcal{L}}\boldsymbol{\gamma}\_{\mathcal{B}} \right)\_{\text{min}} \tag{19}$$

$$\mathcal{R}\_{\mathbb{B}} = \left( \left( \left( \,^{\mathrm{L}} \beta\_{\mathrm{B}} \right)\_{\mathrm{max}} - \left( \,^{\mathrm{L}} \beta\_{\mathrm{B}} \right)\_{\mathrm{min}} \right) \pi \cdot \frac{1}{n} \sum\_{i=1}^{n} \,^{\mathrm{L}} \gamma\_{\mathrm{B}} \right) / 180 \tag{20}$$

where <sup>L</sup>γ<sup>B</sup> � � max <sup>∈</sup> G, <sup>L</sup>γ<sup>B</sup> � � min <sup>∈</sup> G, <sup>L</sup>β<sup>B</sup> � � max <sup>∈</sup> G, <sup>L</sup>β<sup>B</sup> � � min ∈ G. It can be seen from Figure 16 that the angle measurement accuracy is better than the distance measurement accuracy. Therefore, based on these results, this chapter proposes an optimal trilateral localization algorithm based on angle measurement. The idea is to use the azimuth

angle β<sup>b</sup> of the reflector in the platform frame to find the optima γ<sup>b</sup> by the cosine theorem. The global pose of the mobile manipulator is then calculated based on the triangular method. The

First, it is assumed that the finder can cover at least three reflectors at each position. After

same finder, three circles must intersect at the same point <sup>G</sup>OMP , and the value of

According to the cosine theorem, the relations between the above variables can be expressed

B2a � 2P<sup>γ</sup> B1aγB2a cos <sup>P</sup>βB1 � <sup>P</sup>βB2 � � <sup>¼</sup> ð Þ x1 � x2

B3a � 2P<sup>γ</sup> B2aγB3a cos <sup>P</sup>β<sup>R</sup> � <sup>P</sup>βB3 � � <sup>¼</sup> ð Þ x2 � x3

In Eq. (21), the known parameters <sup>P</sup>βB1, <sup>P</sup>βB2, and <sup>P</sup>βB3 are used to solve <sup>P</sup>γB1a , <sup>P</sup>γB2a , and <sup>P</sup>γB3a . Since the above equations are nonlinear, the steepest descent method is adopted. The three

<sup>P</sup><sup>γ</sup> B3a cos <sup>P</sup>βB1 � <sup>P</sup>βB3 � � <sup>¼</sup> ð Þ x1 � x3

GyMP � �, represents the position of the mobile platform, as shown in Figure 17.

� �, respectively. In the global frame, these positions can also

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

http://dx.doi.org/10.5772/intechopen.79598

167

� � . Since they are measured from the

<sup>2</sup> <sup>þ</sup> y1 � y2 � �<sup>2</sup>

<sup>2</sup> <sup>þ</sup> y2 � y3 � �<sup>2</sup>

<sup>2</sup> <sup>þ</sup> y1 � y3 � �<sup>2</sup>

<sup>P</sup>βB1 � �;

(21)

feature extraction, the positions of three reflectors B1, B2, and B3 are calculated as <sup>P</sup>γB1

� � ; and GB3 x3; y3

; <sup>P</sup>βR,B3

� �; GB2 x2; y2

B3a � 2P<sup>γ</sup> B1a

algorithm details are given as follows.

Figure 17. Schematic diagram of the localization algorithm.

<sup>P</sup>γR,B2

GOMP

; <sup>P</sup>βR,B2 � �; and <sup>P</sup>γR,B3

GxMP

be obtained as GB1 x1; y1

by the following equations:

8 >>>>><

>>>>>:

<sup>P</sup>γ<sup>2</sup>

<sup>P</sup>γ<sup>2</sup>

<sup>P</sup>γ<sup>2</sup>

B1a <sup>þ</sup> <sup>P</sup>γ<sup>2</sup>

B2a <sup>þ</sup> <sup>P</sup>γ<sup>2</sup>

B1a <sup>þ</sup> <sup>P</sup>γ<sup>2</sup>

circle equations can be expressed as

Figure 15. Effect diagram using different feature extraction algorithms.

Figure 16. The position of single reflector in the laser range finder frame.

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration http://dx.doi.org/10.5772/intechopen.79598 167

Figure 17. Schematic diagram of the localization algorithm.

It can be seen from Figure 16 that the angle measurement accuracy is better than the distance measurement accuracy. Therefore, based on these results, this chapter proposes an optimal trilateral localization algorithm based on angle measurement. The idea is to use the azimuth

Figure 15. Effect diagram using different feature extraction algorithms.

166 Applications of Mobile Robots

Figure 16. The position of single reflector in the laser range finder frame.

angle β<sup>b</sup> of the reflector in the platform frame to find the optima γ<sup>b</sup> by the cosine theorem. The global pose of the mobile manipulator is then calculated based on the triangular method. The algorithm details are given as follows.

First, it is assumed that the finder can cover at least three reflectors at each position. After feature extraction, the positions of three reflectors B1, B2, and B3 are calculated as <sup>P</sup>γB1 <sup>P</sup>βB1 � �; <sup>P</sup>γR,B2 ; <sup>P</sup>βR,B2 � �; and <sup>P</sup>γR,B3 ; <sup>P</sup>βR,B3 � �, respectively. In the global frame, these positions can also be obtained as GB1 x1; y1 � �; GB2 x2; y2 � � ; and GB3 x3; y3 � � . Since they are measured from the same finder, three circles must intersect at the same point <sup>G</sup>OMP , and the value of GOMP GxMP GyMP � �, represents the position of the mobile platform, as shown in Figure 17.

According to the cosine theorem, the relations between the above variables can be expressed by the following equations:

<sup>P</sup>γ<sup>2</sup> B1a <sup>þ</sup> <sup>P</sup>γ<sup>2</sup> B2a � 2P<sup>γ</sup> B1aγB2a cos <sup>P</sup>βB1 � <sup>P</sup>βB2 � � <sup>¼</sup> ð Þ x1 � x2 <sup>2</sup> <sup>þ</sup> y1 � y2 � �<sup>2</sup> <sup>P</sup>γ<sup>2</sup> B2a <sup>þ</sup> <sup>P</sup>γ<sup>2</sup> B3a � 2P<sup>γ</sup> B2aγB3a cos <sup>P</sup>β<sup>R</sup> � <sup>P</sup>βB3 � � <sup>¼</sup> ð Þ x2 � x3 <sup>2</sup> <sup>þ</sup> y2 � y3 � �<sup>2</sup> <sup>P</sup>γ<sup>2</sup> B1a <sup>þ</sup> <sup>P</sup>γ<sup>2</sup> B3a � 2P<sup>γ</sup> B1a <sup>P</sup><sup>γ</sup> B3a cos <sup>P</sup>βB1 � <sup>P</sup>βB3 � � <sup>¼</sup> ð Þ x1 � x3 <sup>2</sup> <sup>þ</sup> y1 � y3 � �<sup>2</sup> 8 >>>>>< >>>>>: (21)

In Eq. (21), the known parameters <sup>P</sup>βB1, <sup>P</sup>βB2, and <sup>P</sup>βB3 are used to solve <sup>P</sup>γB1a , <sup>P</sup>γB2a , and <sup>P</sup>γB3a . Since the above equations are nonlinear, the steepest descent method is adopted. The three circle equations can be expressed as

$$\begin{cases} \left(\mathbf{x}\_{1} - \,^{\mathrm{G}}\mathbf{x}\_{\mathrm{MP}}\right)^{2} + \left(\mathbf{y}\_{1} - \,^{\mathrm{G}}\mathbf{y}\_{\mathrm{MP}}\right)^{2} = \,^{\mathrm{P}}\mathbf{y}\_{\mathrm{B}\_{\mathrm{la}}}^{2} \\\\ \left(\mathbf{x}\_{2} - \,^{\mathrm{G}}\mathbf{x}\_{\mathrm{MP}}\right)^{2} + \left(\mathbf{y}\_{2} - \,^{\mathrm{G}}\mathbf{y}\_{\mathrm{MP}}\right)^{2} = \,^{\mathrm{P}}\mathbf{y}\_{\mathrm{B}\_{\mathrm{la}}}^{2} \\\\ \left(\mathbf{x}\_{3} - \,^{\mathrm{G}}\mathbf{x}\_{\mathrm{MP}}\right)^{2} + \left(\mathbf{y}\_{3} - \,^{\mathrm{G}}\mathbf{y}\_{\mathrm{MP}}\right)^{2} = \,^{\mathrm{P}}\mathbf{y}\_{\mathrm{B}\_{\mathrm{la}}}^{2} \end{cases} \tag{22}$$

During the matching process, the reflectors observed in real time are made to correspond to all reflectors in the last moment in the environment one by one to extract the effective reflectors

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

During localization, owing to the fact that some of the reflectors are obscured by obstacles or confused with a highly reflective metal surface object, the loss of position of the system is observed. To address the above problems, a dynamic tracking algorithm is proposed as shown

After placing n reflectors in the global environment, the system actually observes q reflectors at the tth moment, and the coordinate value of the ith reflector in the platform frame is

; <sup>β</sup><sup>t</sup>�1,Bj

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

<sup>þ</sup> <sup>γ</sup><sup>t</sup>�1,Bj

� � of the jth reflector in the global environment.

� � <sup>0</sup> <sup>≪</sup> <sup>i</sup> <sup>≪</sup> <sup>q</sup> � �. The sampling time of the laser range finder is 0.1 s. Therefore, the

� � is less than <sup>η</sup>, then the ith observed reflector is an effective reflector and is

� �of the jth reflector in the platform frame can

� � of the system at the tð Þ � <sup>1</sup> th moment and the

; <sup>β</sup><sup>t</sup>�1,Bj

sin <sup>β</sup><sup>t</sup>�1,Bj � <sup>γ</sup>t,Bi sin <sup>β</sup><sup>t</sup>�1,Bj � �<sup>2</sup>

� � and the observed value

http://dx.doi.org/10.5772/intechopen.79598

169

� � � � �

≤ η (28)

[20]. The localization can be achieved.

theoretical position value <sup>M</sup><sup>t</sup>�1,j�<sup>1</sup> <sup>γ</sup><sup>t</sup>�1,B<sup>j</sup>

rt�1,Bj cos <sup>β</sup><sup>t</sup>�1,Bj � <sup>γ</sup>t,Bj

� �<sup>2</sup>

Figure 18. Schematic diagram of the dynamic tracking algorithm.

be deduced by the position Ot�1,<sup>R</sup> xt�<sup>1</sup>; yt�<sup>1</sup>

If the difference between the theoretical value <sup>M</sup><sup>t</sup>�1,<sup>j</sup> <sup>γ</sup><sup>t</sup>�1,Bj

considered matched with the reference reflector Bj as follows:

cos βt,Bi

in Figure 18.

Mt,<sup>i</sup> γt,Bi

Mt,<sup>i</sup> γt,Bi

� � r � � �

; βt,Bi

position <sup>G</sup><sup>O</sup> <sup>j</sup> xj; yj

; βt,Bi

The position GOMP GxMP GyMP � � of the system can be obtained in the global frame by solving the above equations. Since the actual position of the reflector in the global environment deviates from the theoretical position, these circles may not intersect at the same point. In order to minimize the difference between the calculated position of the system and the actual position, the least squares estimation principle is applied. Assuming that the coordinate of the reflector is GBi xi; yi � �ð Þ <sup>i</sup> <sup>¼</sup> <sup>1</sup>; <sup>2</sup>;…; <sup>n</sup> , n is the number of reflectors detected by laser range finder. The position value GOMP GxMP GyMP � � of the system is calculated as

$$\left(\mathbf{^G x\_{MP}}\,\mathbf{^G y\_{MP}}\right)^T = \left(\mathbf{A^T A}\right)^{-1}\mathbf{A^T b} \tag{23}$$

where

$$\mathbf{A} = \begin{bmatrix} 2(\mathbf{x}\_1 - \mathbf{x}\_n) & \cdots & 2\left(\mathbf{y}\_1 - \mathbf{y}\_n\right) \\ \vdots & \cdots & \vdots \\ 2(\mathbf{x}\_{n-1} - \mathbf{x}\_n) & \cdots & 2\left(\mathbf{y}\_{n-1} - \mathbf{y}\_n\right) \end{bmatrix} \tag{24}$$

$$\mathbf{b} = \begin{bmatrix} \mathbf{x}\_1^2 - \mathbf{x}\_n^2 + \mathbf{y}\_1^2 - \mathbf{y}\_n^2 + \mathbb{P}\boldsymbol{\gamma}\_{\mathbf{B}\_{2a}}^2 - \mathbb{P}\boldsymbol{\gamma}\_{\mathbf{B}\_{1a}}^2 \\ \vdots \\ \mathbf{x}\_{n-1}^2 - \mathbf{x}\_n^2 + \mathbf{y}\_{n-1}^2 - \mathbf{y}\_n^2 + \mathbb{P}\boldsymbol{\gamma}\_{\mathbf{B}\_{2a}}^2 - \mathbb{P}\boldsymbol{\gamma}\_{\mathbf{B}\_{1a}}^2 \end{bmatrix} \tag{25}$$

The posture of the system also includes an azimuth angle <sup>G</sup>θ in the global frame. First, <sup>G</sup>θ<sup>i</sup> is obtained from the ith reflector as

$$\mathbf{^G\boldsymbol{\Theta}\_i} = \arctan\left(\left(\mathbf{y\_i} - \mathbf{^G y\_{MP}}\right) / \left(\mathbf{x\_i} - \mathbf{^G x\_{MP}}\right)\right) - \mathbf{^P\boldsymbol{\mathcal{B}}\_i} \tag{26}$$

The azimuth angle <sup>G</sup>θ of the system is the averaged value from all the reflectors, as in

$$\,^G\Theta \,\, = \frac{1}{\mathbf{n}} \sum\_{i=1}^n \,^G\Theta\_i \,\, \tag{27}$$

The dynamic tracking algorithm is then carried out. Localization of the system is based on landmarks. The system needs to meet the following two conditions to complete real-time localization:


During the matching process, the reflectors observed in real time are made to correspond to all reflectors in the last moment in the environment one by one to extract the effective reflectors [20]. The localization can be achieved.

x1 � GxMP � �<sup>2</sup>

8 >>>>><

>>>>>:

GyMP

A ¼

b ¼

> x2 <sup>1</sup> � <sup>x</sup><sup>2</sup>

<sup>G</sup>θ<sup>i</sup> <sup>¼</sup> arctan yi � GyMP

x2 <sup>n</sup>�<sup>1</sup> � <sup>x</sup><sup>2</sup>

GxMP

The position GOMP

168 Applications of Mobile Robots

reflector is GBi xi; yi

where

finder. The position value GOMP

obtained from the ith reflector as

localization:

x2 � GxMP � �<sup>2</sup>

x3 � GxMP � �<sup>2</sup>

GxMP

GxMP

GyMP

GyMP � �<sup>T</sup> <sup>¼</sup> <sup>A</sup><sup>T</sup><sup>A</sup> � ��<sup>1</sup>

<sup>n</sup> <sup>þ</sup> <sup>y</sup><sup>2</sup>

<sup>n</sup> <sup>þ</sup> y2

The azimuth angle <sup>G</sup>θ of the system is the averaged value from all the reflectors, as in

<sup>G</sup><sup>θ</sup> <sup>¼</sup> <sup>1</sup> n Xn i¼1

i. The system requires real-time localization of the reflector in the environment. ii. The localization system correctly matches the real-time observed reflectors.

The dynamic tracking algorithm is then carried out. Localization of the system is based on landmarks. The system needs to meet the following two conditions to complete real-time

<sup>þ</sup> y1 � GyMP

<sup>þ</sup> y2 � GyMP

<sup>þ</sup> y3 � GyMP

the above equations. Since the actual position of the reflector in the global environment deviates from the theoretical position, these circles may not intersect at the same point. In order to minimize the difference between the calculated position of the system and the actual position, the least squares estimation principle is applied. Assuming that the coordinate of the

2 xð Þ <sup>1</sup> � xn ⋯ 2 y1 � yn

⋮⋯⋮ 2 xð Þ <sup>n</sup>�<sup>1</sup> � xn <sup>⋯</sup> 2 yn�<sup>1</sup> � yn

⋮

<sup>n</sup>�<sup>1</sup> � y2

The posture of the system also includes an azimuth angle <sup>G</sup>θ in the global frame. First, <sup>G</sup>θ<sup>i</sup> is

� �<sup>=</sup> xi � GxMP

<sup>n</sup> <sup>þ</sup> <sup>P</sup><sup>γ</sup> <sup>2</sup>

<sup>n</sup> <sup>þ</sup> <sup>P</sup>γ<sup>2</sup>

<sup>1</sup> � y2

� �<sup>2</sup> <sup>¼</sup> <sup>P</sup>γ<sup>2</sup>

� �<sup>2</sup> <sup>¼</sup> <sup>P</sup>γ<sup>2</sup>

� �<sup>2</sup> <sup>¼</sup> <sup>P</sup>γ<sup>2</sup>

� � of the system can be obtained in the global frame by solving

� �ð Þ <sup>i</sup> <sup>¼</sup> <sup>1</sup>; <sup>2</sup>;…; <sup>n</sup> , n is the number of reflectors detected by laser range

� �

� �

B2a � <sup>P</sup>γ<sup>2</sup> B1a

> B2a � <sup>P</sup>γ<sup>2</sup> B1a

� � � � � <sup>P</sup>βBi (26)

<sup>G</sup>θ<sup>i</sup> (27)

� � of the system is calculated as

B1a

B2a

(22)

(24)

(25)

B3a

A<sup>T</sup>b (23)

During localization, owing to the fact that some of the reflectors are obscured by obstacles or confused with a highly reflective metal surface object, the loss of position of the system is observed. To address the above problems, a dynamic tracking algorithm is proposed as shown in Figure 18.

After placing n reflectors in the global environment, the system actually observes q reflectors at the tth moment, and the coordinate value of the ith reflector in the platform frame is Mt,<sup>i</sup> γt,Bi ; βt,Bi � � <sup>0</sup> <sup>≪</sup> <sup>i</sup> <sup>≪</sup> <sup>q</sup> � �. The sampling time of the laser range finder is 0.1 s. Therefore, the theoretical position value <sup>M</sup><sup>t</sup>�1,j�<sup>1</sup> <sup>γ</sup><sup>t</sup>�1,B<sup>j</sup> ; <sup>β</sup><sup>t</sup>�1,Bj � �of the jth reflector in the platform frame can be deduced by the position Ot�1,<sup>R</sup> xt�<sup>1</sup>; yt�<sup>1</sup> � � of the system at the tð Þ � <sup>1</sup> th moment and the position <sup>G</sup><sup>O</sup> <sup>j</sup> xj; yj � � of the jth reflector in the global environment.

If the difference between the theoretical value <sup>M</sup><sup>t</sup>�1,<sup>j</sup> <sup>γ</sup><sup>t</sup>�1,Bj ; <sup>β</sup><sup>t</sup>�1,Bj � � and the observed value Mt,<sup>i</sup> γt,Bi ; βt,Bi � � is less than <sup>η</sup>, then the ith observed reflector is an effective reflector and is considered matched with the reference reflector Bj as follows:

$$\left| \sqrt{\left(\mathbf{r}\_{\mathbf{t}-1,\mathbf{B}\_{\parallel}}\cos\beta\_{\mathbf{t}-1,\mathbf{B}\_{\parallel}} - \gamma\_{\mathbf{t},\mathbf{B}\_{\parallel}}\cos\beta\_{\mathbf{t},\mathbf{B}\_{\parallel}}\right)^{2} + \left(\gamma\_{\mathbf{t}-1,\mathbf{B}\_{\parallel}}\sin\beta\_{\mathbf{t}-1,\mathbf{B}\_{\parallel}} - \gamma\_{\mathbf{t},\mathbf{B}\_{\parallel}}\sin\beta\_{\mathbf{t}-1,\mathbf{B}\_{\parallel}}\right)^{2}} \right| \leq \eta \tag{28}$$

Figure 18. Schematic diagram of the dynamic tracking algorithm.

Therefore, a set A of effective reflectors at the tth moment can be obtained, and A ¼ Mt,<sup>o</sup> γt,Bo βt,Bo � �;…;Mt,<sup>i</sup> <sup>γ</sup>t,Bo ; βt,Bi � �j<sup>i</sup> <sup>¼</sup> <sup>0</sup>; <sup>1</sup>;…; <sup>q</sup> n o. Taking the mobile manipulator'<sup>s</sup> moving speed into account, the value of η depends on the actual situation. Through the use of the optimal triangulation localization algorithm based on angle measurement, the pose Ot,<sup>R</sup> xt; yt � � of the mobile manipulator can be calculated at the tth moment.

outside of the reflector is wrapped by reflective tape. In the experimental environment, five reflectors are placed around the system. Their global coordinate values are (0,995); (0, 0);

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

http://dx.doi.org/10.5772/intechopen.79598

171

The optimal triangulation method based on angle measurement is used for validation by the repeatability of the system. In the stationary state of the system, the environment is scanned by

(0, 1774); (2905, 2449); and (3956, 2032), and the unit is mm, as shown in Figure 20.

Figure 21. Repeatability localization of the system at the same location.

Figure 22. System localization error.

#### 4.3. Experimental verification

The experimental data are obtained by the LMS 100 laser range finder with a scanning range of 270� and an angular resolution of 0.25�. The experimental platform is shown in Figure 19. The

Figure 19. Experimental platform.

Figure 20. Experimental environment.

outside of the reflector is wrapped by reflective tape. In the experimental environment, five reflectors are placed around the system. Their global coordinate values are (0,995); (0, 0); (0, 1774); (2905, 2449); and (3956, 2032), and the unit is mm, as shown in Figure 20.

The optimal triangulation method based on angle measurement is used for validation by the repeatability of the system. In the stationary state of the system, the environment is scanned by

Figure 21. Repeatability localization of the system at the same location.

Therefore, a set A of effective reflectors at the tth moment can be obtained, and

moving speed into account, the value of η depends on the actual situation. Through the use of the optimal triangulation localization algorithm based on angle measurement, the pose

The experimental data are obtained by the LMS 100 laser range finder with a scanning range of 270� and an angular resolution of 0.25�. The experimental platform is shown in Figure 19. The

ji ¼ 0; 1;…; q

. Taking the mobile manipulator's

; βt,Bi � �

� � of the mobile manipulator can be calculated at the tth moment.

A ¼ Mt,<sup>o</sup> γt,Bo

170 Applications of Mobile Robots

Ot,<sup>R</sup> xt; yt

βt,Bo � �

4.3. Experimental verification

Figure 19. Experimental platform.

Figure 20. Experimental environment.

;…;Mt,<sup>i</sup> γt,Bo

n o

Figure 22. System localization error.

finders. Each result is indicated by a red dot in Figure 21. The repeatability obtained by the trilateral method is nearly 18 mm, while the repeatability of the optimal method is only 9 mm. It can be shown that the optimal method is better than the traditional method.

parameters in the motion equation of the Mecanum motion platform is solved by Monte Carlo analysis and interval analysis. Using the relative variation of the parameters to revise the motion equation, the displacement errors in different spaces in theory are solved for and compared with the measured displacement errors. From the comparison, both the methods are found to satisfy the system's requirement. Then, the feasibility of a tracking and locating algorithm for mobile manipulator is demonstrated. The following conclusions can be drawn from this study: (i) In the detection of a reflector in the laser range finder frame, the angle repeatability of the reflector is better than that of the distance repeatability based on the feature extraction algorithm; (ii) The repeatability localization accuracy using the optimal triangulation method based on the angle measurement is nearly 9 mm, which is better than that of the trilateral method; (iii) The localization error of the system is 18 mm, which satisfies the localization requirement of system. Improvements in the localization method based on reflectors, such as optimizing the layout of reflectors and the map of reflectors selection strategy for

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

http://dx.doi.org/10.5772/intechopen.79598

173

The method in this chapter is also used in the research of MoMaCo (Mobile manipulator in Construction), which can draw baseline for architectural decoration engineering as shown in

Figure 24. The application result also verified the effectiveness of the method.

localization, are still needed.

Author details

Tao Song1,2\*, Fengfeng (Jeff) Xi3 and Shuai Guo1,2

Figure 24. MoMaCo (mobile manipulator in construction).

\*Address all correspondence to: songtao43467226@shu.edu.cn

1 Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronics

3 Department of Aerospace Engineering, Ryerson University, Toronto, Ontario, Canada

2 Shanghai Robot Industrial Technology Research Institute, Shanghai, PR China

Engineering and Automation of Shanghai University, Shanghai, PR China

The mobile manipulator moves in the direction of the arrow in Figure 19, and each time the system moves a certain distance, the localization system will perform an experiment, i.e., it will use the left rear finder to calculate the current position. An average of 30 samples is taken for each experiment.

Figure 22 shows the results of static localization accuracy. The maximum distance error is 18 mm and the maximum angle error is 2, which satisfies the localization requirement of the system.

The mobile manipulator moves in the designated route, and it needs to constantly calculate and record its own position in the moving process. As shown in Figure 23, the trajectory of the moving system based on the localization method is smoother.

This chapter demonstrates the feasibility of a tracking and localization algorithm for mobile manipulators. The following conclusions can be drawn from this study: (i) In the detection of a reflector in the laser range finder frame, the angle repeatability of the reflector is better than that of the distance repeatability based on the feature extraction algorithm; (ii) The repeatability localization accuracy using the optimal triangulation method based on the angle measurement is nearly 9 mm, which is better than that of the trilateral method; (iii) The localization error of the system is 18 mm, which satisfies the localization requirement of system. Improvements in the location method based on reflectors, such as optimizing the layout of reflectors and the map of reflectors selection strategy for localization, are still needed.

Figure 23. Tracking results.

#### 5. Summary

In this chapter, through analyzing the roller deformation of the Mecanum wheel, the changed parameters of the motion equation of mobile system are found. The relative variation of the parameters in the motion equation of the Mecanum motion platform is solved by Monte Carlo analysis and interval analysis. Using the relative variation of the parameters to revise the motion equation, the displacement errors in different spaces in theory are solved for and compared with the measured displacement errors. From the comparison, both the methods are found to satisfy the system's requirement. Then, the feasibility of a tracking and locating algorithm for mobile manipulator is demonstrated. The following conclusions can be drawn from this study: (i) In the detection of a reflector in the laser range finder frame, the angle repeatability of the reflector is better than that of the distance repeatability based on the feature extraction algorithm; (ii) The repeatability localization accuracy using the optimal triangulation method based on the angle measurement is nearly 9 mm, which is better than that of the trilateral method; (iii) The localization error of the system is 18 mm, which satisfies the localization requirement of system. Improvements in the localization method based on reflectors, such as optimizing the layout of reflectors and the map of reflectors selection strategy for localization, are still needed.

The method in this chapter is also used in the research of MoMaCo (Mobile manipulator in Construction), which can draw baseline for architectural decoration engineering as shown in Figure 24. The application result also verified the effectiveness of the method.

Figure 24. MoMaCo (mobile manipulator in construction).

#### Author details

finders. Each result is indicated by a red dot in Figure 21. The repeatability obtained by the trilateral method is nearly 18 mm, while the repeatability of the optimal method is only 9 mm.

The mobile manipulator moves in the direction of the arrow in Figure 19, and each time the system moves a certain distance, the localization system will perform an experiment, i.e., it will use the left rear finder to calculate the current position. An average of 30 samples is taken for

Figure 22 shows the results of static localization accuracy. The maximum distance error is 18 mm and the maximum angle error is 2, which satisfies the localization requirement of the system. The mobile manipulator moves in the designated route, and it needs to constantly calculate and record its own position in the moving process. As shown in Figure 23, the trajectory of the

This chapter demonstrates the feasibility of a tracking and localization algorithm for mobile manipulators. The following conclusions can be drawn from this study: (i) In the detection of a reflector in the laser range finder frame, the angle repeatability of the reflector is better than that of the distance repeatability based on the feature extraction algorithm; (ii) The repeatability localization accuracy using the optimal triangulation method based on the angle measurement is nearly 9 mm, which is better than that of the trilateral method; (iii) The localization error of the system is 18 mm, which satisfies the localization requirement of system. Improvements in the location method based on reflectors, such as optimizing the layout of reflectors

In this chapter, through analyzing the roller deformation of the Mecanum wheel, the changed parameters of the motion equation of mobile system are found. The relative variation of the

It can be shown that the optimal method is better than the traditional method.

moving system based on the localization method is smoother.

and the map of reflectors selection strategy for localization, are still needed.

each experiment.

172 Applications of Mobile Robots

5. Summary

Figure 23. Tracking results.

Tao Song1,2\*, Fengfeng (Jeff) Xi3 and Shuai Guo1,2

\*Address all correspondence to: songtao43467226@shu.edu.cn

1 Key Laboratory of Intelligent Manufacturing and Robotics, School of Mechatronics Engineering and Automation of Shanghai University, Shanghai, PR China


#### References

[1] Guo S, Tao S, Xi F(J), Mohamed RP. Tip-over stability analysis for a wheeled mobile manipulator. Journal of Dynamic Systems, Measurement, and Control. 2017;139:054501

[16] Huang L et al. Design and analysis of a four-wheel omnidirectional mobile robot. In: 2nd

Path Tracking of a Wheeled Mobile Manipulator through Improved Localization and Calibration

http://dx.doi.org/10.5772/intechopen.79598

175

[17] Song JB, Byun KS. Design and control of a four-wheeled omnidirectional mobile robot with steerable omnidirectional wheels. Journal of Robotic Systems. 2004;21(4):193-208 [18] Asama H et al. Development of an omni-directional mobile robot with 3 DOF decoupling drive mechanism. In: IEEE International Conference on Robotics and Automation

[19] Liu YM. Three-dimensional optical positioning method research [dissertation]. Shenyang

[20] Jia W, Zhou W, Kaiser J. Efficient algorithm for mobile multicast using any cast group. IEE

International Conference of Autonomous Robots and Agents; 2004

University of Technology; 2003

Proceedings: Communications. 2001;148(1):14-18


[16] Huang L et al. Design and analysis of a four-wheel omnidirectional mobile robot. In: 2nd International Conference of Autonomous Robots and Agents; 2004

References

174 Applications of Mobile Robots

1205-1210

131-143

1987. pp. 1772-1778

[1] Guo S, Tao S, Xi F(J), Mohamed RP. Tip-over stability analysis for a wheeled mobile manipulator. Journal of Dynamic Systems, Measurement, and Control. 2017;139:054501

[2] Tao S, Xi F(J), Guo S, Lin Y. Optimization of a mobile platform for a wheeled manipulator.

[3] Tao S, Xi F(J), Guo S, Tu X, Li X. Slip analysis for a wheeled mobile manipulator. Journal of

[4] Guo S, Jin Y, Bao S, et al. Accuracy analysis of omnidirectional mobile manipulator with

[5] Guo S, Fang T-T, Tao S, Xi F-F, Wei B-G. Tracking and localization for omnidirectional mobile industrial robot using reflectors. Advances in Manufacturing. 2018;6(1):118-125

[6] Larsson U, Forsberg J, Wernersson A. Mobile robot localization: Integrating measurements from a time-of-flight laser. IEEE Transactions on Industrial Electronics. 1996;43(3):422-431

[7] Buschka P, Saffiotti A, Wasik Z. Fuzzy landmark-based localization for a legged robot. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 2; 2000. pp.

[8] Jang G, Lee S, Kweon I. Color landmark based self-localization for indoor mobile robots. In: IEEE International Conference on Robotics & Automation; 2002. pp. 1037-1042

[9] Madsen CB, Andersen CS. Optimal reflector selection for triangulation of robot position.

[10] Betke M, Gurvits L. Mobile localization using landmarks. IEEE Transactions on Robotics

[11] Kai OA, Tomatis N, Jensen BT, et al. Multi-laser range finder on-the-fly localization: Precision and reliability for applications. Robotics and Autonomous Systems. 2001;34(2–3):

[12] Muir PF, Neuman CP. Kinematic modeling for feedback control of an omnidirectional wheeled mobile robot. In: IEEE International Conference on Robotics and Automation;

[13] Wang Y, Chang D. Preferably of characteristics and layout structure of Mecanum compre-

[14] Shimada A, Yajima S, Viboonchaicheep P, et al. Mecanum-wheel vehicle systems based on position corrective control. In: IEEE Annual Conference on Industrial Electronics Society; 2005

[15] Qian J, Wang X, Xia G. Velocity rectification for mecanum wheeled omni-direction robot. Machine Tool and Manufacturing Technology. 2010;11:42-45. http://en.cnki.com.cn/Arti-

hensive system. Journal of Mechanical Engineering. 2009;45(5):307-310

Dynamic Systems, Measurement, and Control. 2017;140(2):021005-021005-12

Journal of Mechanisms and Robotics. 2016;8:061007

Mecanum wheels. Advances in Manufacturing. 2016;4:363

Robotics and Autonomous Systems. 1998;23(4):277-292

and Automation. 1997;13(2):251-263

cle\_en/CJFDTotal-ZJYC201011014.htm


**Chapter 9**

Provisional chapter

**4WD Robot Posture Estimation by Radial Multi-View**

DOI: 10.5772/intechopen.79130

This chapter presents a four-wheel robot's trajectory tracking model by an extended Kalman filter (EKF) estimator for visual odometry using a divergent trinocular visual sensor. The trinocular sensor is homemade and a specific observer model was developed to measure 3D key-points by combining multi-view cameras. The observer approaches a geometric model and the key-points are used as references for estimating the robot's displacement. The robot's displacement is estimated by triangulation of multiple pairs of environmental 3D key-points. The four-wheel drive (4WD) robot's inverse/direct kinematic control law is combined with the visual observer, the visual odometry model, and the EKF. The robot's control law is used to produce experimental locomotion statistical variances and is used as a prediction model in the EKF. The proposed dead-reckoning approach models the four asynchronous drives and the four damping suspensions. This chapter presents the deductions of models, formulations and their validation, as well as the experimental results on posture state estimation comparing the four-wheel deadreckoning model, the visual observer, and the EKF with an external global positioning

Keywords: 4WD, visual odometry, trinocular sensor, EKF, visual observer,

Autonomous robots obtain precise information about their surroundings by deploying their sensing devices and developing perceptual tasks to accomplish useful missions. Intelligent robots require to concurrently execute multiple functions such as path planning, collision avoidance,

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

4WD Robot Posture Estimation by Radial Multi-View

**Visual Odometry**

Visual Odometry

Luz Abril Torres-Méndez

Luz Abril Torres-Méndez

Abstract

reference.

1. Introduction

trajectory estimation

Edgar Alonso Martínez-García and

Edgar Alonso Martínez-García and

http://dx.doi.org/10.5772/intechopen.79130

Additional information is available at the end of the chapter

Additional information is available at the end of the chapter

#### **4WD Robot Posture Estimation by Radial Multi-View Visual Odometry** 4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

DOI: 10.5772/intechopen.79130

Edgar Alonso Martínez-García and Luz Abril Torres-Méndez Edgar Alonso Martínez-García and Luz Abril Torres-Méndez

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79130

#### Abstract

This chapter presents a four-wheel robot's trajectory tracking model by an extended Kalman filter (EKF) estimator for visual odometry using a divergent trinocular visual sensor. The trinocular sensor is homemade and a specific observer model was developed to measure 3D key-points by combining multi-view cameras. The observer approaches a geometric model and the key-points are used as references for estimating the robot's displacement. The robot's displacement is estimated by triangulation of multiple pairs of environmental 3D key-points. The four-wheel drive (4WD) robot's inverse/direct kinematic control law is combined with the visual observer, the visual odometry model, and the EKF. The robot's control law is used to produce experimental locomotion statistical variances and is used as a prediction model in the EKF. The proposed dead-reckoning approach models the four asynchronous drives and the four damping suspensions. This chapter presents the deductions of models, formulations and their validation, as well as the experimental results on posture state estimation comparing the four-wheel deadreckoning model, the visual observer, and the EKF with an external global positioning reference.

Keywords: 4WD, visual odometry, trinocular sensor, EKF, visual observer, trajectory estimation

#### 1. Introduction

Autonomous robots obtain precise information about their surroundings by deploying their sensing devices and developing perceptual tasks to accomplish useful missions. Intelligent robots require to concurrently execute multiple functions such as path planning, collision avoidance,

© 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2018 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

self-localization, tasks scheduling, trajectory control, map building, environment recognition, kinematic/dynamic control, and so forth. Autonomous robots depend on multisensor fusion, which is the process of combining data from the physical sensors into a homogeneous data space.

This chapter is organized into to the following sections. Section 2 deduces the sensor fusion observer modeling the trinocular system geometry. Section 3 models the 4WD direct/inverse kinematic solutions. Section 4 deduces the visual odometry formulation and EKF-based control state and estimation. Finally, conclusions are provided in Sec-

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

http://dx.doi.org/10.5772/intechopen.79130

This section describes the divergent multi-view geometric model, which basically combines the data of a pair of cameras radially arranged. In addition, this section presents an algebraic analysis of the lateral cameras' alignment and correction w.r.t. the central camera. The fundamental geometrical relationship of the system divergence was experimentally studied by deploying a homemade prototype onboard a mobile robot, see Figure 2a. Cameras with homogeneous intrinsic parameters are assumed, and cameras are mechanically fixed epipolar. The sensor model's purpose is to determine the depth information of a point in the scene p ¼ ð Þ x; y; z

which is projected onto the overlapping area of a divergent pair. The proposed multi-view geometric model combines data using the central camera as the reference (Figure 1b). The focal plane in cameras A, C are perspective transformed, in order to align them epipolar and coplanar w.r.t. the central reference B. As seen in Figure 1b, a point p is projected over two focal planes, for instance, at column xC of lateral camera C, and at xB of camera B. Thus, an isosceles triangle PBC is formed. For the triangle OBC, let β be the angle between the cameras' centers B and C, as deployed by expression (1). Let ϕ=2 be the remaining angle of OBC, where the inner angles' total

sum is <sup>π</sup>. By expressing <sup>β</sup> <sup>þ</sup> <sup>ϕ</sup> <sup>¼</sup> <sup>π</sup> and dropping off <sup>ϕ</sup> <sup>¼</sup> <sup>π</sup> � <sup>β</sup>, we easily deduce that <sup>ϕ</sup>

The geometrical distance BC is calculated by triangulation using the law of sines, with known distance l that converges to O. The linear distance between adjacent sensors BC commonly

> BC <sup>¼</sup> <sup>l</sup> sin <sup>β</sup> sin <sup>π</sup> <sup>2</sup> � <sup>β</sup> 2

Figure 2. 4WD kinematics. (a) Deployed robot. (b) Damping system. (c) Robot's view from below.

⊤,

179

<sup>2</sup> <sup>¼</sup> <sup>π</sup> <sup>2</sup> � <sup>β</sup> 2.

: (1)

tion 5.

2. Trinocular sensing model

oriented w.r.t. the center O is

This chapter presents robot's visual odometry using sensor data obtained from a homemade radial multi-view device (Figure 1a). For this case, trinocular sensing is divergent; hence, an inherent problem refers to different perspectives in each camera. Besides, the partial overlap between adjacent cameras allows sharing approximately 25% of the total sensing angles, which is too reduced and limits extracting numerous relevant features for data fusion affecting to infer consistent information. Besides perspective, radial cameras yield differences of scale, skew, rotation, and lighting intensities. To cope with this problem, this chapter deduces a geometric trinocular sensor model to directly measure 3D data by combining divergent pairs, the central camera with one of the two lateral cameras (Figure 1b). The robot's state vector (posture) is recursively estimated by a visual odometry model that triangulates multiple pairs of key-points. Thus, an EKF uses the 3D odometry model and estimates the robot's position. The mobile robot is a four-wheel drive (4WD) modeled by a differential control law involving the four passive damping suspensions to infer accurate positions.

Parallel trinocular stereo systems had been deployed either to detect the ground [1], or to estimate motion [2]. There are reported works on motion estimation with binocular divergent systems [3], trinocular divergence for visual odometry [4], and divergent visual simultaneous localization and mapping (SLAM) [5]. As a difference from the active sensing modalities for localization [6], and concurrent localization and mapping with parallel multi-view [7], this chapter intends to estimate the posture of a rolling vehicle by exploiting feedback of the rich data fusion that a divergent trinocular sensor provides. Numerous visual odometry algorithms had been reported, using stereo cameras [8], matching multi-frame features [9] and 3D point cloud [10]. Some outdoor visual odometry approaches for urban [11] environments estimate motion tracking by extraction of visual feature points. There are numerous works combining the benefits of visual SLAM algorithms [12–14] with visual odometry [15], detecting geometrical features [16].

Figure 1. Robot's trinocular sensor. (a) Trinocular sensor onboard. (b) Camera's geometric divergence (top view).

This chapter is organized into to the following sections. Section 2 deduces the sensor fusion observer modeling the trinocular system geometry. Section 3 models the 4WD direct/inverse kinematic solutions. Section 4 deduces the visual odometry formulation and EKF-based control state and estimation. Finally, conclusions are provided in Section 5.

#### 2. Trinocular sensing model

self-localization, tasks scheduling, trajectory control, map building, environment recognition, kinematic/dynamic control, and so forth. Autonomous robots depend on multisensor fusion, which is the process of combining data from the physical sensors into a homogeneous data space. This chapter presents robot's visual odometry using sensor data obtained from a homemade radial multi-view device (Figure 1a). For this case, trinocular sensing is divergent; hence, an inherent problem refers to different perspectives in each camera. Besides, the partial overlap between adjacent cameras allows sharing approximately 25% of the total sensing angles, which is too reduced and limits extracting numerous relevant features for data fusion affecting to infer consistent information. Besides perspective, radial cameras yield differences of scale, skew, rotation, and lighting intensities. To cope with this problem, this chapter deduces a geometric trinocular sensor model to directly measure 3D data by combining divergent pairs, the central camera with one of the two lateral cameras (Figure 1b). The robot's state vector (posture) is recursively estimated by a visual odometry model that triangulates multiple pairs of key-points. Thus, an EKF uses the 3D odometry model and estimates the robot's position. The mobile robot is a four-wheel drive (4WD) modeled by a differential control law involving

Parallel trinocular stereo systems had been deployed either to detect the ground [1], or to estimate motion [2]. There are reported works on motion estimation with binocular divergent systems [3], trinocular divergence for visual odometry [4], and divergent visual simultaneous localization and mapping (SLAM) [5]. As a difference from the active sensing modalities for localization [6], and concurrent localization and mapping with parallel multi-view [7], this chapter intends to estimate the posture of a rolling vehicle by exploiting feedback of the rich data fusion that a divergent trinocular sensor provides. Numerous visual odometry algorithms had been reported, using stereo cameras [8], matching multi-frame features [9] and 3D point cloud [10]. Some outdoor visual odometry approaches for urban [11] environments estimate motion tracking by extraction of visual feature points. There are numerous works combining the benefits of visual SLAM algorithms [12–14] with visual odometry [15], detecting geometri-

Figure 1. Robot's trinocular sensor. (a) Trinocular sensor onboard. (b) Camera's geometric divergence (top view).

the four passive damping suspensions to infer accurate positions.

cal features [16].

178 Applications of Mobile Robots

This section describes the divergent multi-view geometric model, which basically combines the data of a pair of cameras radially arranged. In addition, this section presents an algebraic analysis of the lateral cameras' alignment and correction w.r.t. the central camera. The fundamental geometrical relationship of the system divergence was experimentally studied by deploying a homemade prototype onboard a mobile robot, see Figure 2a. Cameras with homogeneous intrinsic parameters are assumed, and cameras are mechanically fixed epipolar. The sensor model's purpose is to determine the depth information of a point in the scene p ¼ ð Þ x; y; z ⊤, which is projected onto the overlapping area of a divergent pair. The proposed multi-view geometric model combines data using the central camera as the reference (Figure 1b). The focal plane in cameras A, C are perspective transformed, in order to align them epipolar and coplanar w.r.t. the central reference B. As seen in Figure 1b, a point p is projected over two focal planes, for instance, at column xC of lateral camera C, and at xB of camera B. Thus, an isosceles triangle PBC is formed. For the triangle OBC, let β be the angle between the cameras' centers B and C, as deployed by expression (1). Let ϕ=2 be the remaining angle of OBC, where the inner angles' total sum is <sup>π</sup>. By expressing <sup>β</sup> <sup>þ</sup> <sup>ϕ</sup> <sup>¼</sup> <sup>π</sup> and dropping off <sup>ϕ</sup> <sup>¼</sup> <sup>π</sup> � <sup>β</sup>, we easily deduce that <sup>ϕ</sup> <sup>2</sup> <sup>¼</sup> <sup>π</sup> <sup>2</sup> � <sup>β</sup> 2. The geometrical distance BC is calculated by triangulation using the law of sines, with known distance l that converges to O. The linear distance between adjacent sensors BC commonly oriented w.r.t. the center O is

$$\overline{\mathbf{BC}} = \frac{l \sin \beta}{\sin \left(\frac{\pi}{2} - \frac{\beta}{2}\right)}.\tag{1}$$

Figure 2. 4WD kinematics. (a) Deployed robot. (b) Damping system. (c) Robot's view from below.

To calculate the Cartesian coordinates of p, let us state that the point p is projected through the horizontal coordinate xB, and on camera B angle's θB, and focal distance f <sup>B</sup> as expressed by

$$
\theta\_{\mathcal{B}} = \tan^{-1} \left( \frac{\mathbf{x}\_{\mathcal{B}}}{\mathbf{f}\_{\mathcal{B}}} \right) \quad \text{and} \quad \theta\_{\mathcal{C}} = \tan^{-1} \left( \frac{\mathbf{x}\_{\mathcal{C}}}{\mathbf{f}\_{\mathcal{C}}} \right). \tag{2}
$$

and being dC ¼ zC tan ð Þ θ<sup>C</sup> , by substituting θ<sup>C</sup> from expression (2) we have

dC <sup>¼</sup> zC tan tan �<sup>1</sup> xC

hB <sup>¼</sup> zByB fB

> 0 @

l sin β sin <sup>π</sup> <sup>2</sup> � <sup>β</sup> 2 � �

terms Ψ, zB, and zC in robot's inertial frame <sup>R</sup>, produce the next expression:

pR BC ¼

pR AB ¼

or C. A point coordinates w.r.t. camera A is <sup>p</sup>AB <sup>¼</sup> xA; yA; <sup>z</sup> � �<sup>Τ</sup>

xC; yC; <sup>z</sup> � �<sup>Τ</sup>

for coordinate z,

order to express a general formula, let us define the following theorem.

x, yA,C ¼

zA,C ¼

Ψf 2 <sup>B</sup> sin <sup>π</sup>þ<sup>β</sup>

and hCf<sup>C</sup> ¼ zCyC, w.r.t p using distances hB and hC, is obtained by

<sup>Ψ</sup> <sup>¼</sup> <sup>1</sup> fB

thus the following term is stated as

fC

Furthermore, the algebraic deduction along the Y component for the equalities hBf<sup>B</sup> ¼ zByB

1

Therefore, the geometry vector model for camera B w.r.t. camera C, with substitution of the

sin <sup>θ</sup><sup>B</sup> <sup>þ</sup> <sup>θ</sup><sup>C</sup> � <sup>β</sup> � �

and the same model is enhanced for camera B, using the geometry of cameras A and B by

sin <sup>θ</sup><sup>A</sup> <sup>þ</sup> <sup>θ</sup><sup>B</sup> � <sup>β</sup> � �

Hence, the arbitrary point pABC ∈ ℝ<sup>3</sup> is projected onto cameras AB, or onto cameras BC. In

Theorem 1 (Trinocular depth model). Let camera B be the reference for either divergent camera A

. Hence, the general depth coordinate model for x and y for any divergent pair is

ΨxB sin <sup>π</sup>þ<sup>β</sup>

Ψ sin <sup>π</sup>þ<sup>β</sup>

<sup>2</sup> � θ<sup>C</sup> � �

<sup>2</sup> � θ<sup>A</sup> � �

Ψ sin <sup>π</sup>þ<sup>β</sup>

� � � � or dC <sup>¼</sup> zCxC

and hC <sup>¼</sup> zCyC

fC ,

fB � � � � :

> xB yB fB

xB yB fB 1

0 B@

<sup>2</sup> � θA,C � �

<sup>2</sup> � θA,C � �

sin <sup>θ</sup>A,C <sup>þ</sup> <sup>θ</sup><sup>B</sup> � <sup>β</sup> � � , (12)

sin <sup>θ</sup>A,C <sup>þ</sup> <sup>θ</sup><sup>B</sup> � <sup>β</sup> � � : (13)

1

0 B@

<sup>A</sup> cos tan �<sup>1</sup> xB

fC

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

: (9)

181

http://dx.doi.org/10.5772/intechopen.79130

CA (10)

CA: (11)

, and w.r.t. camera C is pAC ¼

The complementary angles B and C are modeled by

$$
\omega B = \frac{\pi}{2} - \theta\_B + \frac{\beta}{2} \quad \text{and} \quad \omega C = \frac{\pi}{2} - \theta\_C + \frac{\beta}{2}. \tag{3}
$$

In the triangle BCO, the angle at point p is obtained by equivalence of similar triangles ∠P ¼ θ<sup>B</sup> þ θ<sup>C</sup> � β. Thus, to estimate the range of the radial system B and C w.r.t. p, the linear distance is calculated by the law of sines:

$$\frac{\overline{\text{BC}}}{\sin \angle P} = \frac{\overline{\text{CP}}}{\sin \angle B} \quad \text{and} \quad \overline{\text{CP}} = \frac{\overline{\text{BC}} \sin \angle B}{\sin \angle P}. \tag{4}$$

Thus, for the other cameras' pair, similar expressions are stated

$$\frac{\overline{\text{BC}}}{\sin \angle P} = \frac{\overline{\text{BP}}}{\sin \angle \mathbb{C}} \quad \text{and } \overline{\text{BP}} = \frac{\overline{\text{BC}} \sin \angle \mathbb{C}}{\sin \angle P} \,. \tag{5}$$

Hence, the model to express depth information is given by zB ¼ BP cos θB. By substituting BP and θB, the model is further specified by

$$\Lambda\_{\rm I} = \frac{l \sin \beta}{\sin \left(\frac{\pi - \beta}{2}\right) \sin \left(\theta\_{\beta} - \theta\_{\odot} - \beta\right)},$$

where,

$$z\_{\mathcal{B}} = \Lambda\_1 \sin\left(\frac{\pi + \beta}{2} - \theta\_{\mathcal{C}}\right) \cos\left(\tan^{-1}\left(\frac{\chi\_{\mathcal{B}}}{\mathfrak{f}\_{\mathcal{B}}}\right)\right). \tag{6}$$

In addition, the range between camera C and p is defined by zC ¼ Cp cos θC. Thus, substituting Cp and θC, we define

$$z\_{\mathbb{C}} = \Lambda\_1 \sin\left(\frac{\pi + \beta}{2} - \theta\_B\right) \cos\left(\tan^{-1}\left(\frac{\chi\_{\mathbb{C}}}{\xi\_{\mathbb{C}}}\right)\right). \tag{7}$$

Using the depth models zB and zC, the distances dB and dC w.r.t. p are estimated, such that dB ¼ zB tan ð Þ θ<sup>B</sup> . Hence,

$$d\_B = z\_B \tan\left(\tan^{-1}\left(\frac{\chi\_B}{\mathbf{f}\_B}\right)\right) \quad \text{or} \quad d\_B = \frac{z\_B \chi\_B}{\mathbf{f}\_B} \tag{8}$$

and being dC ¼ zC tan ð Þ θ<sup>C</sup> , by substituting θ<sup>C</sup> from expression (2) we have

$$d\_{\mathbb{C}} = \mathbf{z}\_{\mathbb{C}} \tan \left( \tan^{-1} \left( \frac{\mathbf{x}\_{\mathbb{C}}}{\mathbf{f}\_{\mathbb{C}}} \right) \right) \quad \text{or} \quad d\_{\mathbb{C}} = \frac{\mathbf{z}\_{\mathbb{C}} \mathbf{x}\_{\mathbb{C}}}{\mathbf{f}\_{\mathbb{C}}}. \tag{9}$$

Furthermore, the algebraic deduction along the Y component for the equalities hBf<sup>B</sup> ¼ zByB and hCf<sup>C</sup> ¼ zCyC, w.r.t p using distances hB and hC, is obtained by

$$h\_B = \frac{z\_B y\_B}{\mathbf{f}\_B} \quad \text{and} \quad h\_\mathbb{C} = \frac{z\_\mathbb{C} y\_\mathbb{C}}{\mathbf{f}\_\mathbb{C}},$$

thus the following term is stated as

To calculate the Cartesian coordinates of p, let us state that the point p is projected through the horizontal coordinate xB, and on camera B angle's θB, and focal distance f <sup>B</sup> as expressed by

<sup>2</sup> and <sup>∠</sup><sup>C</sup> <sup>¼</sup> <sup>π</sup>

sin∠<sup>B</sup> and CP <sup>¼</sup> BC sin <sup>∠</sup><sup>B</sup>

sin <sup>∠</sup><sup>C</sup> and BP <sup>¼</sup> BC sin∠<sup>C</sup>

sin <sup>θ</sup><sup>B</sup> � <sup>θ</sup><sup>C</sup> � <sup>β</sup> ,

cos tan �<sup>1</sup> xB

cos tan �<sup>1</sup> xC

fB 

fC 

or dB <sup>¼</sup> zBxB

fB

Hence, the model to express depth information is given by zB ¼ BP cos θB. By substituting BP

<sup>Λ</sup><sup>1</sup> <sup>¼</sup> <sup>l</sup> sin <sup>β</sup> sin <sup>π</sup>�<sup>β</sup> 2 

> <sup>2</sup> � <sup>θ</sup><sup>C</sup>

> <sup>2</sup> � <sup>θ</sup><sup>B</sup>

In addition, the range between camera C and p is defined by zC ¼ Cp cos θC. Thus, substituting

Using the depth models zB and zC, the distances dB and dC w.r.t. p are estimated, such that

fB 

In the triangle BCO, the angle at point p is obtained by equivalence of similar triangles ∠P ¼ θ<sup>B</sup> þ θ<sup>C</sup> � β. Thus, to estimate the range of the radial system B and C w.r.t. p, the linear

and <sup>θ</sup><sup>C</sup> <sup>¼</sup> tan �<sup>1</sup> xC

fC 

2

<sup>2</sup> � <sup>θ</sup><sup>C</sup> <sup>þ</sup> <sup>β</sup>

: (2)

: (3)

sin∠<sup>P</sup> : (4)

sin <sup>∠</sup><sup>P</sup> : (5)

: (6)

: (7)

(8)

<sup>θ</sup><sup>B</sup> <sup>¼</sup> tan �<sup>1</sup> xB

The complementary angles B and C are modeled by

distance is calculated by the law of sines:

180 Applications of Mobile Robots

and θB, the model is further specified by

where,

Cp and θC, we define

dB ¼ zB tan ð Þ θ<sup>B</sup> . Hence,

<sup>∠</sup><sup>B</sup> <sup>¼</sup> <sup>π</sup>

BC sin∠<sup>P</sup> <sup>¼</sup> CP

Thus, for the other cameras' pair, similar expressions are stated

zB <sup>¼</sup> <sup>Λ</sup><sup>1</sup> sin <sup>π</sup> <sup>þ</sup> <sup>β</sup>

zC <sup>¼</sup> <sup>Λ</sup><sup>1</sup> sin <sup>π</sup> <sup>þ</sup> <sup>β</sup>

dB <sup>¼</sup> zB tan tan �<sup>1</sup> xB

BC sin <sup>∠</sup><sup>P</sup> <sup>¼</sup> BP

fB 

<sup>2</sup> � <sup>θ</sup><sup>B</sup> <sup>þ</sup> <sup>β</sup>

$$\Psi = \frac{1}{\mathbf{f}\_B} \left( \frac{l \sin \beta}{\sin \left( \frac{\pi}{2} - \frac{\beta}{2} \right)} \right) \cos \left( \tan^{-1} \left( \frac{\chi\_B}{\mathbf{f}\_B} \right) \right).$$

Therefore, the geometry vector model for camera B w.r.t. camera C, with substitution of the terms Ψ, zB, and zC in robot's inertial frame <sup>R</sup>, produce the next expression:

$$\mathbf{p}\_{\rm BC}^{\rm R} = \frac{\Psi \sin\left(\frac{\pi + \beta}{2} - \theta\_{\rm C}\right)}{\sin\left(\theta\_{\rm B} + \theta\_{\rm C} - \beta\right)} \begin{pmatrix} \mathbf{x}\_{\rm B} \\ \mathbf{y}\_{\rm B} \\ \mathbf{f}\_{\rm B} \end{pmatrix} \tag{10}$$

and the same model is enhanced for camera B, using the geometry of cameras A and B by

$$\mathbf{p}\_{AB}^{\mathbb{R}} = \frac{\Psi \sin\left(\frac{\pi + \beta}{2} - \Theta\_A\right)}{\sin\left(\Theta\_A + \Theta\_B - \beta\right)} \begin{pmatrix} \mathbf{x}\_B\\ \mathbf{y}\_B\\ \mathbf{f}\_B \end{pmatrix}. \tag{11}$$

Hence, the arbitrary point pABC ∈ ℝ<sup>3</sup> is projected onto cameras AB, or onto cameras BC. In order to express a general formula, let us define the following theorem.

Theorem 1 (Trinocular depth model). Let camera B be the reference for either divergent camera A or C. A point coordinates w.r.t. camera A is <sup>p</sup>AB <sup>¼</sup> xA; yA; <sup>z</sup> � �<sup>Τ</sup> , and w.r.t. camera C is pAC ¼ xC; yC; <sup>z</sup> � �<sup>Τ</sup> . Hence, the general depth coordinate model for x and y for any divergent pair is

$$\chi, y\_{A, \mathbb{C}} = \frac{\Psi \chi\_{\mathbb{B}} \sin \left( \frac{n + \beta}{2} - \Theta\_{A, \mathbb{C}} \right)}{\sin \left( \Theta\_{A, \mathbb{C}} + \Theta\_{\mathbb{B}} - \beta \right)},\tag{12}$$

for coordinate z,

$$z\_{A,\mathcal{C}} = \frac{\Psi f\_B^2 \sin\left(\frac{\pi+\beta}{2} - \Theta\_{A,\mathcal{C}}\right)}{\sin\left(\Theta\_{A,\mathcal{C}} + \Theta\_B - \beta\right)}.\tag{13}$$

The four points shown by the three cameras may illustrate their transformation, experimentally developed at 1-m distance between the robot and the marks.

<sup>ω</sup><sup>t</sup> <sup>¼</sup> <sup>v</sup>^ cos cos ð Þ ð Þ <sup>α</sup><sup>i</sup> li

The previous equation expresses the conservation of angular motion, and the wheel's contact

W 2li

q

q

cos ð Þ¼ α<sup>i</sup>

<sup>2</sup> , l<sup>2</sup> <sup>¼</sup>

<sup>2</sup> , l<sup>4</sup> <sup>¼</sup>

Thus, substituting cos ð Þ α<sup>i</sup> and li into ω<sup>t</sup> and by considering both clockwise and counterclock-

The longitudinal contact point's distance li (m) takes as reference the robot's geometric center.

Li ¼ d cos arcsin γ<sup>i</sup>

γ<sup>i</sup> ¼ arcsin

my€<sup>d</sup> þ κ2y\_

From Figure 2a, the vertical motion yd is modeled assuming critical damping motion for a general spring-mass-damper system. The suspension is modeled by the following second-

The restitution force my€<sup>d</sup> counteracts the vertical oscillatory damping effects. The oscillatory

�<sup>X</sup> 4

i¼3

Δy d1

rWφ\_ <sup>i</sup> l 2 i

rWφ\_ <sup>i</sup> l 2 i

point turns w.r.t. the robot's center,

as well as

where for each length li there is an asynchronous model,

q

q

When li varies, the contact point's position Li changes.

where γ<sup>i</sup> represents the vertical motion along the suspension,

where the elastic spring restitution coefficient is κ<sup>1</sup> (kg/s<sup>2</sup>

order differential equation as a first-order equation such that κ2y\_

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi <sup>W</sup><sup>2</sup> <sup>þ</sup> ð Þ <sup>L</sup><sup>1</sup> <sup>þ</sup> <sup>L</sup><sup>4</sup> 2 2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi <sup>W</sup><sup>2</sup> <sup>þ</sup> ð Þ <sup>L</sup><sup>3</sup> <sup>þ</sup> <sup>L</sup><sup>2</sup> 2 2

> <sup>ω</sup><sup>t</sup> <sup>¼</sup> <sup>X</sup> 2

> > i¼1

l<sup>1</sup> ¼

l<sup>3</sup> ¼

wise motions, the robot's angular speed is

order homogeneous differential equation:

velocity and acceleration are denoted by y\_

: (17)

http://dx.doi.org/10.5772/intechopen.79130

183

, (18)

: (19)

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi <sup>W</sup><sup>2</sup> <sup>þ</sup> ð Þ <sup>L</sup><sup>2</sup> <sup>þ</sup> <sup>L</sup><sup>3</sup> 2 2

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi <sup>W</sup><sup>2</sup> <sup>þ</sup> ð Þ <sup>L</sup><sup>4</sup> <sup>þ</sup> <sup>L</sup><sup>1</sup> 2 2

<sup>2</sup> ,

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

<sup>2</sup> :

� � � � , (20)

� �: (21)

<sup>d</sup> þ κ1yd ¼ 0, (22)

<sup>d</sup> and y€d, respectively. Thus, solving the second-

<sup>d</sup> ¼ �κ1yd,

). The damping coefficient is κ<sup>2</sup> (kg/s).

#### 3. 4WD dead-reckoning controller

Since the visual trinocular approach uses an exteroceptive sensor, we decided to challenge its detection and tracking capabilities with a robot having high holonomic properties. A 4WD robot's locomotion is prone to experience frequent swift turns resulting in numerous slippages. Thus, a 4WD has to depend more on exteroceptive rather than inner measurements. Comparatively, inner 4WD odometry differs greatly from external visual measurement to infer posture. The proposed dead-reckoning system obtains speed measurements by deploying odometer readings of the four asynchronous drives (Figure 2). A 4WD system is considerably different from a conventional differential dual approach. Moreover, four passive mass-springdamper suspensions are included in this system (Figure 2b), which varies the inter-wheel distances over time. Particularly, the robot's 4WD and passive suspensions make the posture observations challenging.

The robot's dead-reckoning model is fundamental to sense and control position used as feedback, providing motion description as a kinematic reference to match the visual observations when estimating the robot's motion. The positioning and trajectory control [17], as well as the type of kinematic analysis [18] and the dynamic suspension [19] in this type of robot have been previously reported. The robot's instantaneous speed vt (m/s) and yaw rate ω<sup>t</sup> (rad/s) depend on the four wheels' asynchronous rolling motion, φ\_ <sup>1</sup>,φ\_ <sup>2</sup>,φ\_ <sup>3</sup>,φ\_ <sup>4</sup>. For a wheel's encoder, the velocity model approaches measurements by high-precision numerical derivatives (central divided differences) of the rotary angle φ<sup>t</sup> (rad) w.r.t. time t, such that

$$\phi\_t = \frac{\pi}{6Rt} \left[ \eta\_{t+2} + 7\eta\_{t+1} + 7\phi\_{t-1} - \eta\_{t-2} \right] \tag{14}$$

where the wheel's angular speed φ\_ is measured through pulse detection η (dimensionless) of encoder's resolution R (pulses/rev); thus, the robot's instantaneous velocity is modeled by the averaged wheel speed, with wheels of nominal radius r (m)

$$
v\_t = \frac{r}{4} \sum\_{i=1}^{4} \phi\_i.\tag{15}$$

Further, the differential velocity v^<sup>t</sup> expresses the lateral speeds' difference that yields ωt. Thus, v^<sup>t</sup> is formulated by the expression

$$
\hat{\upsilon}\_t = r(\dot{\varphi}\_1 + \dot{\varphi}\_2 - \dot{\varphi}\_3 - \dot{\varphi}\_4). \tag{16}
$$

This model describes that the rotary motion of φ\_ <sup>1</sup> and φ\_ <sup>2</sup> contributes to robot's þω<sup>t</sup> (counterclockwise sense). Likewise, φ\_ <sup>3</sup> and φ\_ <sup>4</sup> contribute to robot's �ω<sup>t</sup> (clockwise sense). Therefore, ω<sup>t</sup> is yielded by the lateral speed component v^<sup>t</sup> cos ð Þ α<sup>i</sup> (see Figure 2b) modeled by

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry http://dx.doi.org/10.5772/intechopen.79130 183

$$
\omega\_t = \frac{\hat{v}\cos\left(\cos\left(\alpha\_i\right)\right)}{l\_i}.\tag{17}
$$

The previous equation expresses the conservation of angular motion, and the wheel's contact point turns w.r.t. the robot's center,

$$\cos\left(\alpha\_i\right) = \frac{W}{2l\_i},\tag{18}$$

where for each length li there is an asynchronous model,

$$l\_1 = \frac{\sqrt[2]{\mathcal{W}^2 + \left(L\_1 + L\_4\right)^2}}{2}, \quad l\_2 = \frac{\sqrt[2]{\mathcal{W}^2 + \left(L\_2 + L\_3\right)^2}}{2}.$$

as well as

The four points shown by the three cameras may illustrate their transformation, experimen-

Since the visual trinocular approach uses an exteroceptive sensor, we decided to challenge its detection and tracking capabilities with a robot having high holonomic properties. A 4WD robot's locomotion is prone to experience frequent swift turns resulting in numerous slippages. Thus, a 4WD has to depend more on exteroceptive rather than inner measurements. Comparatively, inner 4WD odometry differs greatly from external visual measurement to infer posture. The proposed dead-reckoning system obtains speed measurements by deploying odometer readings of the four asynchronous drives (Figure 2). A 4WD system is considerably different from a conventional differential dual approach. Moreover, four passive mass-springdamper suspensions are included in this system (Figure 2b), which varies the inter-wheel distances over time. Particularly, the robot's 4WD and passive suspensions make the posture

The robot's dead-reckoning model is fundamental to sense and control position used as feedback, providing motion description as a kinematic reference to match the visual observations when estimating the robot's motion. The positioning and trajectory control [17], as well as the type of kinematic analysis [18] and the dynamic suspension [19] in this type of robot have been previously reported. The robot's instantaneous speed vt (m/s) and yaw rate ω<sup>t</sup> (rad/s) depend on the four wheels' asynchronous rolling motion, φ\_ <sup>1</sup>,φ\_ <sup>2</sup>,φ\_ <sup>3</sup>,φ\_ <sup>4</sup>. For a wheel's encoder, the velocity model approaches measurements by high-precision numerical derivatives (central

<sup>6</sup>Rt <sup>η</sup><sup>t</sup>þ<sup>2</sup> <sup>þ</sup> <sup>7</sup>η<sup>t</sup>þ<sup>1</sup> <sup>þ</sup> <sup>7</sup>φ<sup>t</sup>�<sup>1</sup> � <sup>η</sup><sup>t</sup>�<sup>2</sup>

i¼1 φ\_ i

Further, the differential velocity v^<sup>t</sup> expresses the lateral speeds' difference that yields ωt. Thus,

v^<sup>t</sup> ¼ r φ\_ <sup>1</sup> þ φ\_ <sup>2</sup> � φ\_ <sup>3</sup> � φ\_ <sup>4</sup>

This model describes that the rotary motion of φ\_ <sup>1</sup> and φ\_ <sup>2</sup> contributes to robot's þω<sup>t</sup> (counterclockwise sense). Likewise, φ\_ <sup>3</sup> and φ\_ <sup>4</sup> contribute to robot's �ω<sup>t</sup> (clockwise sense). Therefore,

ω<sup>t</sup> is yielded by the lateral speed component v^<sup>t</sup> cos ð Þ α<sup>i</sup> (see Figure 2b) modeled by

where the wheel's angular speed φ\_ is measured through pulse detection η (dimensionless) of encoder's resolution R (pulses/rev); thus, the robot's instantaneous velocity is modeled by the

> vt <sup>¼</sup> <sup>r</sup> 4 X 4

� �, (14)

� �: (16)

: (15)

tally developed at 1-m distance between the robot and the marks.

divided differences) of the rotary angle φ<sup>t</sup> (rad) w.r.t. time t, such that

<sup>φ</sup>\_ <sup>t</sup> <sup>¼</sup> <sup>π</sup>

averaged wheel speed, with wheels of nominal radius r (m)

v^<sup>t</sup> is formulated by the expression

3. 4WD dead-reckoning controller

observations challenging.

182 Applications of Mobile Robots

$$l\_3 = \frac{\sqrt[2]{\mathcal{W}^2 + \left(L\_3 + L\_2\right)^2}}{2}, \quad l\_4 = \frac{\sqrt[2]{\mathcal{W}^2 + \left(L\_4 + L\_1\right)^2}}{2}.$$

Thus, substituting cos ð Þ α<sup>i</sup> and li into ω<sup>t</sup> and by considering both clockwise and counterclockwise motions, the robot's angular speed is

$$
\omega\_l = \sum\_{i=1}^{2} \frac{r\mathcal{W}\dot{\phi}\_i}{l\_i^2} - \sum\_{i=3}^{4} \frac{r\mathcal{W}\dot{\phi}\_i}{l\_i^2}.\tag{19}
$$

The longitudinal contact point's distance li (m) takes as reference the robot's geometric center. When li varies, the contact point's position Li changes.

$$L\_i = d \cos\left(\arcsin(\gamma\_i)\right),\tag{20}$$

where γ<sup>i</sup> represents the vertical motion along the suspension,

$$\gamma\_i = \arcsin\left(\frac{\Delta\_y}{d\_1}\right). \tag{21}$$

From Figure 2a, the vertical motion yd is modeled assuming critical damping motion for a general spring-mass-damper system. The suspension is modeled by the following secondorder homogeneous differential equation:

$$
\kappa \ddot{y}\_d + \kappa\_2 \dot{y}\_d + \kappa\_1 y\_d = 0,\tag{22}
$$

where the elastic spring restitution coefficient is κ<sup>1</sup> (kg/s<sup>2</sup> ). The damping coefficient is κ<sup>2</sup> (kg/s). The restitution force my€<sup>d</sup> counteracts the vertical oscillatory damping effects. The oscillatory velocity and acceleration are denoted by y\_ <sup>d</sup> and y€d, respectively. Thus, solving the secondorder differential equation as a first-order equation such that κ2y\_ <sup>d</sup> ¼ �κ1yd,

$$\int\_{y\_d} \frac{\mathbf{d}y\_d}{y\_d} = -\frac{\kappa\_1}{\kappa\_2} \int\_t \mathbf{d}t,\tag{23}$$

<sup>x</sup>\_<sup>R</sup> <sup>¼</sup> rW 4vmax

R

0

BBBBB@

of independent control rotary variables <sup>Ω</sup>\_ <sup>t</sup> <sup>¼</sup> <sup>φ</sup>€1;φ€2;φ€3;φ€<sup>4</sup>

<sup>t</sup> � u\_ <sup>t</sup> or

y\_ <sup>R</sup> <sup>¼</sup> rL 4vmax

� �<sup>⊤</sup>

control vector is u\_ <sup>R</sup> ¼ v\_t; ω\_ <sup>t</sup>; x\_R; y\_

2

v\_t ω\_ t x\_R y\_ R 1

CCCCCA ¼

0

BBBBB@

λ<sup>1</sup> ¼ r=4, λ2<sup>i</sup> ¼ rW=l

λ<sup>F</sup> ¼ λ<sup>22</sup> � λ23.

and

variations is stated as <sup>Ω</sup>\_ <sup>¼</sup> <sup>Λ</sup>�<sup>1</sup>

solution is <sup>u</sup>\_ <sup>R</sup> <sup>¼</sup> <sup>Λ</sup><sup>t</sup> � <sup>Ω</sup>\_ or

and

φ€<sup>1</sup> � φ€<sup>2</sup> � φ€<sup>3</sup> þ φ€<sup>4</sup>

�φ€<sup>1</sup> � φ€<sup>2</sup> þ φ€<sup>3</sup> þ φ€<sup>4</sup>

<sup>i</sup> , λ<sup>3</sup> ¼ rW=ð Þ 4vmax , and λ<sup>4</sup> ¼ rL=ð Þ 4vmax . Thus, the forward kinematics

� �<sup>⊤</sup>

1

CCCCCA �

There is a maximal allowable Z-turn displacement speed vmax. Hereafter, with four independent equations, the control positioning system is instantaneously computed. The robot

> λ<sup>1</sup> λ<sup>1</sup> λ<sup>1</sup> λ<sup>1</sup> λ<sup>21</sup> λ<sup>22</sup> �λ<sup>23</sup> �λ<sup>24</sup> λ<sup>3</sup> �λ<sup>3</sup> �λ<sup>3</sup> λ<sup>3</sup> �λ<sup>4</sup> �λ<sup>4</sup> λ<sup>4</sup> λ<sup>4</sup>

In addition, to inversely solve this matrix system, the analytical solution represents the vector

λ1λ3λ4, λ<sup>A</sup> ¼ λ<sup>23</sup> � λ24, λ<sup>B</sup> ¼ λ<sup>22</sup> � λ21, λ<sup>C</sup> ¼ λ<sup>21</sup> � λ23, λ<sup>D</sup> ¼ λ<sup>24</sup> � λ22, λ<sup>E</sup> ¼ λ<sup>21</sup> þ λ24, and

<sup>φ</sup>€<sup>1</sup> <sup>¼</sup> <sup>λ</sup>3λ4λDv\_ <sup>þ</sup> <sup>2</sup>λwω\_ � <sup>λ</sup>1λ4λAx\_<sup>R</sup> <sup>þ</sup> <sup>λ</sup>1λ3λFy\_

<sup>φ</sup>€<sup>2</sup> <sup>¼</sup> <sup>λ</sup>3λ4λCv\_ � <sup>2</sup>λwω\_ <sup>þ</sup> <sup>λ</sup>1λ4λAx\_<sup>R</sup> � <sup>λ</sup>1λ3λEy\_

<sup>φ</sup>€<sup>3</sup> <sup>¼</sup> <sup>λ</sup>3λ4λDv\_ <sup>þ</sup> <sup>2</sup>λwω\_ <sup>þ</sup> <sup>λ</sup>1λ4λBx\_<sup>R</sup> <sup>þ</sup> <sup>λ</sup>1λ3λEy\_

<sup>φ</sup>€<sup>4</sup> <sup>¼</sup> <sup>λ</sup>3λ4λCv\_ � <sup>2</sup>λwω\_ � <sup>λ</sup>1λ4λBx\_<sup>R</sup> � <sup>λ</sup>1λ3λFy\_

The matrix form of the inverse analytical solution for all wheels' speed under damping

2λwð Þ λ<sup>21</sup> � λ<sup>22</sup> � λ<sup>23</sup> þ λ<sup>24</sup>

2λwð Þ λ<sup>21</sup> � λ<sup>22</sup> � λ<sup>23</sup> þ λ<sup>24</sup>

2λwð Þ λ<sup>21</sup> � λ<sup>22</sup> � λ<sup>23</sup> þ λ<sup>24</sup>

2λwð Þ λ<sup>21</sup> � λ<sup>22</sup> � λ<sup>23</sup> þ λ<sup>24</sup>

� � (31)

http://dx.doi.org/10.5772/intechopen.79130

185

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

� �: (32)

, and the control transition matrix Λ<sup>t</sup> has the elements

φ€1 φ€2 φ€3 φ€4 1

CCCCCA

R

R

R

R

: (33)

. Thus, let us define λ<sup>w</sup> ¼

, (34)

, (35)

: (37)

(36)

0

BBBBB@

hence

$$\ln\left(y\_d\right) = -\frac{\kappa\_1}{\kappa\_2}t + c, \quad \zeta = -\frac{\kappa\_1}{\kappa\_2},\tag{24}$$

with integration constant c ¼ 0 for analysis purpose. The suspension's elongation derivatives as functions of time are

$$\mathbf{y}\_d = \mathbf{e}^{\zeta t}, \quad \dot{\mathbf{y}}\_d = \zeta \mathbf{e}^{\zeta t}, \quad \ddot{\mathbf{y}}\_d = \zeta^2 \mathbf{e}^{\zeta t}. \tag{25}$$

Substituting the previous expression in (22),

$$m\zeta^2 \mathbf{e}^{\check{\zeta}t} + \kappa\_2 \check{\zeta} \mathbf{e}^{\check{\zeta}t} + \kappa\_1 \mathbf{e}^{\check{\zeta}t} = \mathbf{0},\tag{26}$$

and by algebraically simplifying, the characteristic equation is

$$
\zeta^2 + \frac{\kappa\_2}{m}\zeta + \frac{\kappa\_1}{m} = 0,\tag{27}
$$

and its analytic solution is

$$\zeta\_{1,2} = \frac{-\varkappa\_2}{2m} \pm \frac{\sqrt[2]{\left(\frac{\varkappa\_1}{m}\right)^2 - 4\frac{\varkappa\_1}{m}}}{2}.\tag{28}$$

As we assume a critically damped system, <sup>κ</sup><sup>2</sup> m � �<sup>2</sup> <sup>¼</sup> <sup>4</sup> <sup>κ</sup><sup>1</sup> m � � and there is only one real root solution, such that

$$
\zeta = -\frac{\kappa\_2}{2m}.\tag{29}
$$

Therefore, the damping motion is analytically solved by

$$y\_d(t) = A e^{\zeta t},\tag{30}$$

where A (m) is the elongation amplitude (m) parameter for the suspension system. Moreover, in this type of robotic platform, the four asynchronous drives simultaneously produce slip/ skid motions that are advantageously used to maneuver the robot. This approach proposes inferring the instantaneous Z-turn axis location xR; yR � �<sup>⊤</sup> . The Z-turn axis is movable in the square region bounded by the wheels' contact point. The Z-turn axis is expressed by the firstorder derivatives

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry http://dx.doi.org/10.5772/intechopen.79130 185

$$
\dot{\mathbf{x}}\_R = \frac{rW}{4\upsilon\_{\text{max}}} \left( \ddot{\boldsymbol{\varphi}}\_1 - \ddot{\boldsymbol{\varphi}}\_2 - \ddot{\boldsymbol{\varphi}}\_3 + \ddot{\boldsymbol{\varphi}}\_4 \right) \tag{31}
$$

and

ð yd

� � ¼ � <sup>κ</sup><sup>1</sup>

, y\_

<sup>ζ</sup><sup>2</sup> <sup>þ</sup> κ2 m ζ þ κ1

<sup>ζ</sup>1,<sup>2</sup> <sup>¼</sup> �κ<sup>2</sup> 2m �

ln yd

yd <sup>¼</sup> <sup>e</sup><sup>ζ</sup><sup>t</sup>

mζ<sup>2</sup>

and by algebraically simplifying, the characteristic equation is

hence

as functions of time are

184 Applications of Mobile Robots

and its analytic solution is

such that

order derivatives

Substituting the previous expression in (22),

As we assume a critically damped system, <sup>κ</sup><sup>2</sup>

Therefore, the damping motion is analytically solved by

inferring the instantaneous Z-turn axis location xR; yR

dyd yd

κ2

with integration constant c ¼ 0 for analysis purpose. The suspension's elongation derivatives

<sup>d</sup> <sup>¼</sup> <sup>ζ</sup>e<sup>ζ</sup><sup>t</sup>

¼ � <sup>κ</sup><sup>1</sup> κ2 ð t

<sup>t</sup> <sup>þ</sup> c, <sup>ζ</sup> ¼ � <sup>κ</sup><sup>1</sup>

, <sup>y</sup>€<sup>d</sup> <sup>¼</sup> <sup>ζ</sup><sup>2</sup>

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi κ1 m � �<sup>2</sup> � <sup>4</sup> <sup>κ</sup><sup>1</sup> m

m

� �<sup>⊤</sup>

2 q

m � �<sup>2</sup> <sup>¼</sup> <sup>4</sup> <sup>κ</sup><sup>1</sup>

<sup>ζ</sup> ¼ � <sup>κ</sup><sup>2</sup>

ydðÞ¼ <sup>t</sup> <sup>A</sup>e<sup>ζ</sup><sup>t</sup>

where A (m) is the elongation amplitude (m) parameter for the suspension system. Moreover, in this type of robotic platform, the four asynchronous drives simultaneously produce slip/ skid motions that are advantageously used to maneuver the robot. This approach proposes

square region bounded by the wheels' contact point. The Z-turn axis is expressed by the first-

κ2

eζt

<sup>e</sup><sup>ζ</sup><sup>t</sup> <sup>þ</sup> <sup>κ</sup>2ζe<sup>ζ</sup><sup>t</sup> <sup>þ</sup> <sup>κ</sup>1e<sup>ζ</sup><sup>t</sup> <sup>¼</sup> <sup>0</sup>, (26)

<sup>m</sup> <sup>¼</sup> <sup>0</sup>, (27)

<sup>2</sup> : (28)

� � and there is only one real root solution,

<sup>2</sup><sup>m</sup> : (29)

, (30)

. The Z-turn axis is movable in the

dt, (23)

, (24)

: (25)

$$
\dot{y}\_R = \frac{rL}{4\upsilon\_{max}} \left( -\ddot{\varphi}\_1 - \ddot{\varphi}\_2 + \ddot{\varphi}\_3 + \ddot{\varphi}\_4 \right). \tag{32}
$$

There is a maximal allowable Z-turn displacement speed vmax. Hereafter, with four independent equations, the control positioning system is instantaneously computed. The robot control vector is u\_ <sup>R</sup> ¼ v\_t; ω\_ <sup>t</sup>; x\_R; y\_ R � �<sup>⊤</sup> , and the control transition matrix Λ<sup>t</sup> has the elements λ<sup>1</sup> ¼ r=4, λ2<sup>i</sup> ¼ rW=l 2 <sup>i</sup> , λ<sup>3</sup> ¼ rW=ð Þ 4vmax , and λ<sup>4</sup> ¼ rL=ð Þ 4vmax . Thus, the forward kinematics solution is <sup>u</sup>\_ <sup>R</sup> <sup>¼</sup> <sup>Λ</sup><sup>t</sup> � <sup>Ω</sup>\_ or

$$
\begin{pmatrix}
\dot{\boldsymbol{\nu}}\_{t} \\
\dot{\boldsymbol{\omega}}\_{t} \\
\dot{\boldsymbol{x}}\_{R} \\
\dot{\boldsymbol{y}}\_{R}
\end{pmatrix} = \begin{pmatrix}
\lambda\_{1} & \lambda\_{1} & \lambda\_{1} & \lambda\_{1} \\
\lambda\_{21} & \lambda\_{22} & -\lambda\_{23} & -\lambda\_{24} \\
& \lambda\_{3} & -\lambda\_{3} & -\lambda\_{3} & \lambda\_{3} \\
& -\lambda\_{4} & -\lambda\_{4} & \lambda\_{4} & \lambda\_{4}
\end{pmatrix} \cdot \begin{pmatrix}
\ddot{\boldsymbol{\varphi}}\_{1} \\
\ddot{\boldsymbol{\varphi}}\_{2} \\
\ddot{\boldsymbol{\varphi}}\_{3} \\
\ddot{\boldsymbol{\varphi}}\_{4}
\end{pmatrix}.
\tag{33}
$$

In addition, to inversely solve this matrix system, the analytical solution represents the vector of independent control rotary variables <sup>Ω</sup>\_ <sup>t</sup> <sup>¼</sup> <sup>φ</sup>€1;φ€2;φ€3;φ€<sup>4</sup> � �<sup>⊤</sup> . Thus, let us define λ<sup>w</sup> ¼ λ1λ3λ4, λ<sup>A</sup> ¼ λ<sup>23</sup> � λ24, λ<sup>B</sup> ¼ λ<sup>22</sup> � λ21, λ<sup>C</sup> ¼ λ<sup>21</sup> � λ23, λ<sup>D</sup> ¼ λ<sup>24</sup> � λ22, λ<sup>E</sup> ¼ λ<sup>21</sup> þ λ24, and λ<sup>F</sup> ¼ λ<sup>22</sup> � λ23.

$$\ddot{\varphi}\_1 = \frac{\lambda\_3 \lambda\_4 \lambda\_D \dot{\upsilon} + 2 \lambda\_w \dot{\omega} - \lambda\_1 \lambda\_4 \lambda\_A \dot{\mathbf{x}}\_R + \lambda\_1 \lambda\_3 \lambda\_F \dot{y}\_R}{2\lambda\_w (\lambda\_{21} - \lambda\_{22} - \lambda\_{23} + \lambda\_{24})},\tag{34}$$

$$\ddot{\boldsymbol{\phi}}\_{2} = \frac{\lambda\_{3}\lambda\_{4}\lambda\_{C}\dot{\boldsymbol{v}} - 2\lambda\_{w}\dot{\boldsymbol{w}} + \lambda\_{1}\lambda\_{4}\lambda\_{A}\dot{\boldsymbol{x}}\_{R} - \lambda\_{1}\lambda\_{3}\lambda\_{E}\dot{\boldsymbol{y}}\_{R}}{2\lambda\_{w}(\lambda\_{21} - \lambda\_{22} - \lambda\_{23} + \lambda\_{24})},\tag{35}$$

$$
\ddot{\varphi}\_3 = \frac{\lambda\_3 \lambda\_4 \lambda\_D \dot{\varphi} + 2\lambda\_w \dot{\omega} + \lambda\_1 \lambda\_4 \lambda\_B \dot{x}\_R + \lambda\_1 \lambda\_3 \lambda\_E \dot{y}\_R}{2\lambda\_w (\lambda\_{21} - \lambda\_{22} - \lambda\_{23} + \lambda\_{24})} \tag{36}
$$

and

$$\ddot{\varphi}\_4 = \frac{\lambda\_3 \lambda\_4 \lambda\_C \dot{v} - 2\lambda\_w \dot{w} - \lambda\_1 \lambda\_4 \lambda\_B \dot{x}\_R - \lambda\_1 \lambda\_3 \lambda\_F \dot{y}\_R}{2\lambda\_w (\lambda\_{21} - \lambda\_{22} - \lambda\_{23} + \lambda\_{24})}.\tag{37}$$

The matrix form of the inverse analytical solution for all wheels' speed under damping variations is stated as <sup>Ω</sup>\_ <sup>¼</sup> <sup>Λ</sup>�<sup>1</sup> <sup>t</sup> � u\_ <sup>t</sup> or

$$
\dot{\Omega}\_{t} = \begin{pmatrix}
\frac{\lambda\_{3}\lambda\_{4}\lambda\_{D}}{2\lambda\_{w}\lambda\_{G}} & \frac{2\lambda\_{w}}{2\lambda\_{w}\lambda\_{G}} & \frac{-\lambda\_{1}\lambda\_{4}\lambda\_{A}}{2\lambda\_{w}\lambda\_{G}} & \frac{\lambda\_{1}\lambda\_{3}\lambda\_{F}}{2\lambda\_{w}\lambda\_{G}} \\
\frac{\lambda\_{3}\lambda\_{4}\lambda\_{C}}{2\lambda\_{w}\lambda\_{G}} & \frac{-2\lambda\_{w}}{2\lambda\_{w}\lambda\_{G}} & \frac{\lambda\_{1}\lambda\_{4}\lambda\_{A}}{2\lambda\_{w}\lambda\_{G}} & \frac{-\lambda\_{1}\lambda\_{3}\lambda\_{E}}{2\lambda\_{w}\lambda\_{G}} \\
\frac{\lambda\_{3}\lambda\_{4}\lambda\_{D}}{2\lambda\_{w}\lambda\_{G}} & \frac{2\lambda\_{w}}{2\lambda\_{w}\lambda\_{G}} & \frac{\lambda\_{1}\lambda\_{4}\lambda\_{B}}{2\lambda\_{w}\lambda\_{G}} & \frac{\lambda\_{1}\lambda\_{3}\lambda\_{E}}{2\lambda\_{w}\lambda\_{G}} \\
\frac{\lambda\_{3}\lambda\_{4}\lambda\_{G}}{2\lambda\_{w}\lambda\_{G}} & \frac{-2\lambda\_{W}}{2\lambda\_{w}\lambda\_{G}} & \frac{-\lambda\_{1}\lambda\_{4}\lambda\_{B}}{2\lambda\_{w}\lambda\_{G}} & \frac{-\lambda\_{1}\lambda\_{3}\lambda\_{F}}{2\lambda\_{w}\lambda\_{G}}
\end{pmatrix},
\begin{pmatrix}
\dot{\boldsymbol{v}}\_{t} \\
\dot{\boldsymbol{w}}\_{t} \\
\dot{\boldsymbol{x}}\_{R} \\
\dot{\boldsymbol{y}}\_{R}
\end{pmatrix},
\tag{38}$$

βa, <sup>b</sup>

ct ¼

that is inferred by triangulation of visual key-points over time is

Δ<sup>s</sup> ¼

The triangulation angle λ is calculated by the law of sines,

and the orientation angle for each reference a is

which is required to know the X displacement

as well as the Y displacement

δa <sup>t</sup>�<sup>1</sup> <sup>þ</sup> <sup>δ</sup><sup>a</sup>

<sup>λ</sup>a, <sup>b</sup> <sup>¼</sup> arcsin

<sup>ϕ</sup>a, <sup>b</sup> <sup>¼</sup> <sup>λ</sup>a, <sup>b</sup> <sup>þ</sup>

<sup>Δ</sup><sup>x</sup> <sup>¼</sup> <sup>Δ</sup>a, <sup>b</sup>

<sup>Δ</sup><sup>y</sup> <sup>¼</sup> <sup>Δ</sup><sup>a</sup>

averaged value of the displacements yielded by all key-point pairs,

with the key-point's distance ct as

which is used to obtain the angle βa, <sup>b</sup>

arbitrary 3D point pa, <sup>b</sup> (Figure 3c),

However, at time t in Figure 3b, the instantaneous angle α<sup>t</sup> is obtained by

δa t � �<sup>2</sup> <sup>þ</sup> <sup>δ</sup><sup>b</sup>

q

βa, <sup>b</sup>

<sup>t</sup>�<sup>1</sup> <sup>¼</sup> arcsin <sup>δ</sup>b, <sup>a</sup>

<sup>α</sup><sup>t</sup> <sup>¼</sup> <sup>∣</sup>θ<sup>a</sup>

<sup>t</sup> <sup>∣</sup> <sup>þ</sup> <sup>∣</sup>θ<sup>b</sup>

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

� <sup>2</sup>δ<sup>a</sup> t δb

<sup>t</sup> of the key-point a or b at actual time

<sup>t</sup> sin α<sup>t</sup> ct !

<sup>t</sup> cos ð Þ α<sup>t</sup>

t � �<sup>2</sup>

<sup>t</sup> <sup>¼</sup> arcsin <sup>δ</sup>b, <sup>a</sup>

<sup>β</sup>^ <sup>¼</sup> <sup>β</sup>a, <sup>b</sup>

Further, the differential angle β^ is defined by triangulation of previous and actual poses and an

Proposition 1 (Triangulation odometric displacement). The robot's displacement Δ<sup>s</sup> (Figure 3c)

<sup>t</sup>�<sup>1</sup> � <sup>β</sup>a, <sup>b</sup>

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

<sup>t</sup>�<sup>1</sup>δ<sup>a</sup>

Δa, <sup>b</sup> s

!

<sup>t</sup> � <sup>2</sup>δ<sup>a</sup>

δa, <sup>b</sup> <sup>t</sup> sin <sup>β</sup>^ � �

π <sup>2</sup> � <sup>θ</sup>a, <sup>b</sup> t�1

<sup>s</sup> sin ϕ<sup>a</sup>

When obtaining numerous key-point pairs simultaneously, the total robot's displacement is an

<sup>t</sup>�<sup>1</sup> sin <sup>α</sup><sup>t</sup>�<sup>1</sup> ct�<sup>1</sup> !

: (41)

http://dx.doi.org/10.5772/intechopen.79130

187

, (43)

: (44)

(47)

<sup>t</sup> : (45)

� �, (48)

<sup>s</sup> cos <sup>ϕ</sup>a, <sup>b</sup> � �, (49)

: (50)

<sup>t</sup> cos <sup>β</sup>^ <sup>q</sup> � �: (46)

<sup>t</sup> ∣, (42)

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

where λ<sup>G</sup> ¼ λ<sup>21</sup> � λ<sup>22</sup> � λ<sup>23</sup> þ λ24.

#### 4. State estimation and feedback position control

This section formulates a deterministic geometric model for visual odometry and the state estimation by an EKF. The proposed model combines pairs of key-points at times t and t � 1. The robot's displacements are deduced by inverse geometric triangulations to feed forward an EKF and estimate the robot's posture.

In Figure 3a, the instantaneous angle α<sup>t</sup>�<sup>1</sup> is formed by a pair of key-points defined by

$$\alpha\_{t-1} = |\theta\_{t-1}^{a}| + |\theta\_{t-1}^{b}| \tag{39}$$

and such key-points' distance ct�<sup>1</sup> is defined by

$$\mathbf{c}\_{t-1} = \sqrt{\left(\delta\_{t-1}^{a}\right)^{2} + \left(\delta\_{t-1}^{b}\right)^{2} - 2\delta\_{t-1}^{a}\delta\_{t-1}^{b}\cos a\_{t-1}}.\tag{40}$$

The angle βa, <sup>b</sup> <sup>t</sup>�<sup>1</sup> of either key-point <sup>a</sup> or <sup>b</sup> is calculated by the law of sines,

Figure 3. Robot's visual odometry. (a) Robot's key-point pair observed at t � 1. (b) Same pair observed at t. (c) Robot's displacement Δ<sup>s</sup> by triangulation of key-points pa.

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry http://dx.doi.org/10.5772/intechopen.79130 187

$$\beta\_{t-1}^{a,b} = \arcsin\left(\frac{\delta\_{t-1}^{b,a}\sin\alpha\_{t-1}}{c\_{t-1}}\right). \tag{41}$$

However, at time t in Figure 3b, the instantaneous angle α<sup>t</sup> is obtained by

$$
\alpha\_t = |\Theta\_t^a| + |\Theta\_t^b|, \tag{42}
$$

with the key-point's distance ct as

<sup>Ω</sup>\_ <sup>t</sup> <sup>¼</sup>

186 Applications of Mobile Robots

where λ<sup>G</sup> ¼ λ<sup>21</sup> � λ<sup>22</sup> � λ<sup>23</sup> þ λ24.

EKF and estimate the robot's posture.

displacement Δ<sup>s</sup> by triangulation of key-points pa.

The angle βa, <sup>b</sup>

and such key-points' distance ct�<sup>1</sup> is defined by

ct�<sup>1</sup> ¼

δa t�1 � �<sup>2</sup> <sup>þ</sup> <sup>δ</sup><sup>b</sup>

q

λ3λ4λ<sup>D</sup> 2λwλ<sup>G</sup>

0

BBBBBBBBBBB@

λ3λ4λ<sup>C</sup> 2λwλ<sup>G</sup>

λ3λ4λ<sup>D</sup> 2λwλ<sup>G</sup>

λ3λ4λ<sup>C</sup> 2λwλ<sup>G</sup>

4. State estimation and feedback position control

2λ<sup>w</sup> 2λwλ<sup>G</sup>

�2λ<sup>w</sup> 2λwλ<sup>G</sup>

2λ<sup>w</sup> 2λwλ<sup>G</sup>

�2λ<sup>w</sup> 2λwλ<sup>G</sup> �λ1λ4λ<sup>A</sup> 2λwλ<sup>G</sup>

λ1λ4λ<sup>A</sup> 2λwλ<sup>G</sup>

λ1λ4λ<sup>B</sup> 2λwλ<sup>G</sup>

�λ1λ4λ<sup>B</sup> 2λwλ<sup>G</sup>

This section formulates a deterministic geometric model for visual odometry and the state estimation by an EKF. The proposed model combines pairs of key-points at times t and t � 1. The robot's displacements are deduced by inverse geometric triangulations to feed forward an

<sup>t</sup>�<sup>1</sup><sup>∣</sup> <sup>þ</sup> <sup>∣</sup>θ<sup>b</sup>

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

� <sup>2</sup>δ<sup>a</sup> <sup>t</sup>�<sup>1</sup>δ<sup>b</sup>

<sup>t</sup>�<sup>1</sup> cos <sup>α</sup><sup>t</sup>�<sup>1</sup>

t�1 � �<sup>2</sup>

Figure 3. Robot's visual odometry. (a) Robot's key-point pair observed at t � 1. (b) Same pair observed at t. (c) Robot's

In Figure 3a, the instantaneous angle α<sup>t</sup>�<sup>1</sup> is formed by a pair of key-points defined by

<sup>α</sup><sup>t</sup>�<sup>1</sup> <sup>¼</sup> <sup>∣</sup>θ<sup>a</sup>

<sup>t</sup>�<sup>1</sup> of either key-point <sup>a</sup> or <sup>b</sup> is calculated by the law of sines,

λ1λ3λ<sup>F</sup> 2λwλ<sup>G</sup>

1

v\_t ω\_ t x\_R y\_ R 1

<sup>t</sup>�<sup>1</sup><sup>∣</sup> (39)

: (40)

CCCA, (38)

0

BBB@

CCCCCCCCCCCA �

�λ1λ3λ<sup>E</sup> 2λwλ<sup>G</sup>

> λ1λ3λ<sup>E</sup> 2λwλ<sup>G</sup>

�λ1λ3λ<sup>F</sup> 2λwλ<sup>G</sup>

$$\mathbf{c}\_{t} = \sqrt{\left(\delta\_{t}^{a}\right)^{2} + \left(\delta\_{t}^{b}\right)^{2} - 2\delta\_{t}^{a}\delta\_{t}^{b}\cos\left(\alpha\_{t}\right)}\tag{43}$$

which is used to obtain the angle βa, <sup>b</sup> <sup>t</sup> of the key-point a or b at actual time

$$\beta\_t^{a,b} = \arcsin\left(\frac{\delta\_t^{b,a}\sin\alpha\_t}{c\_t}\right). \tag{44}$$

Further, the differential angle β^ is defined by triangulation of previous and actual poses and an arbitrary 3D point pa, <sup>b</sup> (Figure 3c),

$$
\hat{\beta} = \beta\_{t-1}^{a,b} - \beta\_t^{a,b}.\tag{45}
$$

Proposition 1 (Triangulation odometric displacement). The robot's displacement Δ<sup>s</sup> (Figure 3c) that is inferred by triangulation of visual key-points over time is

$$
\Delta\_s = \sqrt{\delta\_{t-1}^a + \delta\_t^a - 2\delta\_{t-1}^a \delta\_t^a \cos\left(\hat{\boldsymbol{\beta}}\right)}.\tag{46}
$$

The triangulation angle λ is calculated by the law of sines,

$$\lambda^{a,b} = \arcsin\left(\frac{\delta\_t^{a,b}\sin\left(\hat{\beta}\right)}{\Delta\_s^{a,b}}\right) \tag{47}$$

and the orientation angle for each reference a is

$$
\phi^{a,b} = \lambda^{a,b} + \left(\frac{\pi}{2} - \Theta^{a,b}\_{t-1}\right),
\tag{48}
$$

which is required to know the X displacement

$$
\Delta\_{\mathfrak{x}} = \Delta\_{\mathfrak{s}}^{a,b} \cos \left( \phi^{a,b} \right),
\tag{49}
$$

as well as the Y displacement

$$
\Delta\_{\mathcal{Y}} = \Delta\_s^a \sin \phi^a. \tag{50}
$$

When obtaining numerous key-point pairs simultaneously, the total robot's displacement is an averaged value of the displacements yielded by all key-point pairs,

$$
\Delta\_{\mathbf{x}\_{l}} = \frac{\sum\_{i=1}^{n} \Delta\_{\mathbf{x}\_{i}}}{n} \quad \text{and} \quad \Delta\_{\mathbf{y}\_{l}} = \frac{\sum\_{i=1}^{n} \Delta\_{\mathbf{y}\_{i}}}{n}. \tag{51}
$$

Therefore, without loss of generality, for state estimation, let us assume a nonlinear robot's model state vector

$$\mathbf{x}\_k = f(\mathbf{x}\_{k-1}, \mathbf{u}\_k, \mathbf{w}\_k)\_\prime \tag{52}$$

fð Þ¼ x fð Þþ x^ f

hð Þ¼ x hð Þþ x^ h<sup>0</sup>

x^�

<sup>k</sup> <sup>¼</sup> <sup>A</sup>kP<sup>k</sup>�<sup>1</sup>A<sup>⊤</sup>

<sup>k</sup> H<sup>⊤</sup>

x^<sup>k</sup> ¼ x^�

<sup>k</sup> HkP� <sup>k</sup> H<sup>⊤</sup>

P<sup>k</sup> ¼ ð Þ I � KkH<sup>k</sup> P�

Thus, hereafter, the vector and matrix models describing the proposed robot's system are formulated and incorporated into the conventional EKF. Let us define the robot's pose vector

P�

K<sup>k</sup> ¼ P�

Moreover, the recursive Kalman gain for system convergence is

and the state vector of the system is described by

Eventually, the process noise vector w<sup>k</sup> ¼ wx; wy; w<sup>θ</sup>

with covariance matrix of the system

and

and

x ¼ xt; yt

u<sup>k</sup> ¼ ð Þ υ; ω

vXp ; vYp ; vZp � �<sup>⊤</sup>

and

; θ<sup>t</sup> � �<sup>⊤</sup>

.

the robot's Cartesian displacements are

and linearized as

0 ð Þ x^ |ffl{zffl} ¼J

ð Þ x^ |ffl{zffl} ¼H

<sup>k</sup> <sup>þ</sup> <sup>W</sup>kQ<sup>k</sup>�<sup>1</sup>W<sup>⊤</sup>

<sup>k</sup> <sup>þ</sup> <sup>V</sup>kRkV<sup>⊤</sup>

. The control vector is comprised of the robot's absolute and angular speeds,

π <sup>2</sup> � <sup>θ</sup><sup>a</sup> t�1

<sup>⊤</sup>. Furthermore, the observation vector with sensor measurement <sup>z</sup><sup>k</sup> <sup>¼</sup> Xp;Yp;Zp

� �<sup>⊤</sup>

Therefore, from the displacement equation (46), which arises from exteroceptive observations,

Δx ¼ ð Þ Δs cos θ<sup>k</sup>�<sup>1</sup> þ λ þ

k

In addition, the EKF's prediction models (56) and the correction models (57) are formulated

ð Þ x � x^ (57)

http://dx.doi.org/10.5772/intechopen.79130

189

ð Þ x � x^ : (58)

<sup>k</sup> : (60)

<sup>k</sup> ¼ fð Þ x<sup>k</sup>�<sup>1</sup>; u<sup>k</sup>�<sup>1</sup>; 0 (59)

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

� ��<sup>1</sup> (61)

<sup>k</sup> þ Kkð Þ z<sup>k</sup> � Hx<sup>k</sup> , (62)

� � � � (64)

<sup>k</sup> : (63)

. The measurement noise vector v<sup>k</sup> ¼

� �<sup>⊤</sup>

.

where the state vector is x ¼ ð Þ x; y; θ; v; ω , and combined with a nonstationary state transition matrix At, such that x<sup>k</sup> ¼ A<sup>t</sup> � x<sup>k</sup>�<sup>1</sup> or

$$\mathbf{x}\_{k} = \begin{pmatrix} 1 & 0 & 0 & \cos\left(\theta\right)\Delta t & 0 \\ 0 & 1 & 0 & \sin\left(\theta\right)\Delta t & 0 \\ 0 & 0 & 1 & 0 & \Delta t \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \end{pmatrix} \cdot \begin{pmatrix} x \\ y \\ \theta \\ v \\ w \end{pmatrix} \tag{53}$$

by developing the dot product from previous expression, we obtain

$$\mathbf{x}\_{k} = \begin{pmatrix} \mathbf{x}\_{k-1} + \upsilon \cos \left(\theta \right) \Delta t \\\\ \mathbf{x}\_{k-1} + \upsilon \sin \left(\theta \right) \Delta t \\\\ \theta\_{k-1} + \upsilon \Delta t \\\\ \upsilon \\\\ \omega \end{pmatrix}. \tag{54}$$

The measurement model requires the displacements that were inferred through key-point triangulation

$$\mathbf{z}\_k = h(\mathbf{x}\_k, \mathbf{v}\_k),\tag{55}$$

where w<sup>k</sup> and v<sup>k</sup> are the process and measurement noise models, respectively. These are statistically independent and supposed to have a Gauss distribution with zero average value and known variance. To approximate the nonlinear robot's measurement model, a linearized firstorder approximation by the expansion of the Taylor series is used, and a linear approximation of a function is built, with slope obtained through partial derivatives by

$$f'(\mathbf{u}\_t, \mathbf{x}\_{t-1}) = \frac{\partial f(\mathbf{u}\_t, \mathbf{x}\_{t-1})}{\partial \mathbf{x}\_{t-1}}.\tag{56}$$

Thus, the linearized models of the process and measurement are defined next in (54) and (55), such that

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry http://dx.doi.org/10.5772/intechopen.79130 189

$$f(\mathbf{x}) = f(\hat{\mathbf{x}}) + \underbrace{f'(\hat{\mathbf{x}})}\_{=\mathbf{I}}(\mathbf{x} - \hat{\mathbf{x}}) \tag{57}$$

and

Δxt ¼

x<sup>k</sup> ¼

0

BBBBBB@

by developing the dot product from previous expression, we obtain

x<sup>k</sup> ¼

a function is built, with slope obtained through partial derivatives by

f

<sup>0</sup> ut ð Þ¼ ; xt�<sup>1</sup>

0

BBBBBB@

model state vector

188 Applications of Mobile Robots

triangulation

such that

matrix At, such that x<sup>k</sup> ¼ A<sup>t</sup> � x<sup>k</sup>�<sup>1</sup> or

P<sup>n</sup> <sup>i</sup>¼<sup>1</sup> Δxi n

and Δyt ¼

Therefore, without loss of generality, for state estimation, let us assume a nonlinear robot's

where the state vector is x ¼ ð Þ x; y; θ; v; ω , and combined with a nonstationary state transition

xk�<sup>1</sup> þ v cos ð Þ θ Δt xk�<sup>1</sup> þ v sin ð Þ θ Δt θ<sup>k</sup>�<sup>1</sup> þ ωΔt v ω

The measurement model requires the displacements that were inferred through key-point

where w<sup>k</sup> and v<sup>k</sup> are the process and measurement noise models, respectively. These are statistically independent and supposed to have a Gauss distribution with zero average value and known variance. To approximate the nonlinear robot's measurement model, a linearized firstorder approximation by the expansion of the Taylor series is used, and a linear approximation of

Thus, the linearized models of the process and measurement are defined next in (54) and (55),

∂f ut ð Þ ; xt�<sup>1</sup> ∂xt�<sup>1</sup>

1 0 0 cos ð Þ θ Δt 0 0 1 0 sin ð Þ θ Δt 0 001 0 Δt 000 1 0 000 0 1

P<sup>n</sup> <sup>i</sup>¼<sup>1</sup> Δyi

1

CCCCCCA �

1

CCCCCCA

x<sup>k</sup> ¼ f x<sup>k</sup>�<sup>1</sup>; u<sup>k</sup> ð Þ ; w<sup>k</sup> , (52)

x y θ v ω 1

CCCCCCA

z<sup>k</sup> ¼ h x<sup>k</sup> ð Þ ; v<sup>k</sup> , (55)

0

BBBBBB@

<sup>n</sup> : (51)

, (53)

: (54)

: (56)

$$h(\mathbf{x}) = h(\hat{\mathbf{x}}) + \underbrace{h'(\hat{\mathbf{x}})}\_{=H} (\mathbf{x} - \hat{\mathbf{x}}).\tag{58}$$

In addition, the EKF's prediction models (56) and the correction models (57) are formulated and linearized as

$$
\hat{\mathbf{x}}\_k^- = f(\mathbf{x}\_{k-1}, \mathbf{u}\_{k-1}, \mathbf{0}) \tag{59}
$$

and

$$\mathbf{P}\_k^- = \mathbf{A}\_k \mathbf{P}\_{k-1} \mathbf{A}\_k^\top + \mathbf{W}\_k \mathbf{Q}\_{k-1} \mathbf{W}\_k^\top. \tag{60}$$

Moreover, the recursive Kalman gain for system convergence is

$$\mathbf{K}\_{k} = \mathbf{P}\_{k}^{-} \mathbf{H}\_{k}^{\top} \left( \mathbf{H}\_{k} \mathbf{P}\_{k}^{-} \mathbf{H}\_{k}^{\top} + \mathbf{V}\_{k} \mathbf{R}\_{k} \mathbf{V}\_{k}^{\top} \right)^{-1} \tag{61}$$

and the state vector of the system is described by

$$
\hat{\mathbf{x}}\_k = \hat{\mathbf{x}}\_k^- + \mathbf{K}\_k (\mathbf{z}\_k - \mathbf{H}\overline{\mathbf{x}}\_k),
\tag{62}
$$

with covariance matrix of the system

$$\mathbf{P}\_k = (\mathbf{I} - \mathbf{K}\_k \mathbf{H}\_k) \mathbf{P}\_k^{-}. \tag{63}$$

Thus, hereafter, the vector and matrix models describing the proposed robot's system are formulated and incorporated into the conventional EKF. Let us define the robot's pose vector x ¼ xt; yt ; θ<sup>t</sup> � �<sup>⊤</sup> . The control vector is comprised of the robot's absolute and angular speeds, u<sup>k</sup> ¼ ð Þ υ; ω <sup>⊤</sup>. Furthermore, the observation vector with sensor measurement <sup>z</sup><sup>k</sup> <sup>¼</sup> Xp;Yp;Zp � �<sup>⊤</sup> . Eventually, the process noise vector w<sup>k</sup> ¼ wx; wy; w<sup>θ</sup> � �<sup>⊤</sup> . The measurement noise vector v<sup>k</sup> ¼ vXp ; vYp ; vZp � �<sup>⊤</sup> .

Therefore, from the displacement equation (46), which arises from exteroceptive observations, the robot's Cartesian displacements are

$$
\Delta \mathbf{x} = (\Delta \mathbf{s}) \cos \left( \theta\_{k-1} + \lambda + \left( \frac{\pi}{2} - \theta\_{t-1}^{\mathbf{a}} \right) \right) \tag{64}
$$

and

$$
\Delta y = (\Delta \mathbf{s}) \sin \left( \Theta\_{k-1} + \lambda + \left( \frac{\pi}{2} - \Theta\_{t-1}^a \right) \right), \tag{65}
$$

as well as Δθ is given by

$$
\Delta\theta = \lambda + \left(\frac{\pi}{2} - \Theta\_{t-1}^a\right). \tag{66}
$$

the matrix diagonal variances are experimental measurements that describe the trend of the

The robot's motion covariance matrix was obtained experimentally through 500 tests—straight motion, right turns, left turns, clockwise and counterclockwise rotations, with N ¼ 100 tests for each type of motion. Exteroceptive sensing devices onboard are tied to the robot's geometry of motion, and with their observations the robot's posture can be estimated and therefore matched with the robot's deterministic kinematic model. From Section 3, the inverse (38) and direct (33) models were used experimentally to obtain the following statistical covariance

t

t

�rW l 2 3

xp 0 0 000

yp 0 000

∂xk ∂wy

∂yk ∂wy

∂θ<sup>k</sup> ∂wy

∂xk ∂w<sup>θ</sup> 1

CCCCCCCA

∂yk ∂w<sup>θ</sup>

∂θ<sup>k</sup> ∂w<sup>θ</sup> :

yp 000

!<sup>⊤</sup>

rW l 2 1

rW l 2 1

0 !<sup>2</sup>

rW l 2 2

rW l 2 2

@ (72)

�rW l 2 4

!<sup>2</sup>

@ (71)

�rW l 2 3

!

!<sup>⊤</sup>

�rW l 2 3

!

!<sup>⊤</sup>

�rW l 2 4

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

�rW l 2 4

� <sup>Ω</sup>\_ <sup>d</sup><sup>t</sup> � <sup>θ</sup>

1

� <sup>Ω</sup>\_ <sup>d</sup><sup>t</sup>

http://dx.doi.org/10.5772/intechopen.79130

191

� <sup>Ω</sup>\_ <sup>d</sup><sup>t</sup>

� x

� y !2 ,

: (73)

CCA, (74)

: (75)

<sup>⊤</sup> � <sup>Ω</sup>\_ <sup>d</sup><sup>t</sup> � cos <sup>ð</sup>

<sup>⊤</sup> � <sup>Ω</sup>\_ <sup>d</sup><sup>t</sup> � sin <sup>ð</sup>

rW l 2 1

σ2

0

BB@

0 σ<sup>2</sup>

0 0 σ<sup>2</sup>

and the matrix W<sup>k</sup> which is the partial derivative of the process model w.r.t. the process noise

∂xk ∂wx

0

BBBBBBB@

The matrix V<sup>k</sup> ¼ ∂h=∂v<sup>k</sup> is the partial derivative w.r.t. the measurement noise vector,

∂yk ∂wx

∂θ<sup>k</sup> ∂wx

rW l 2 2

robot motion's error.

σ2 <sup>x</sup> <sup>¼</sup> <sup>1</sup> N X N

and

σ2 <sup>y</sup> <sup>¼</sup> <sup>1</sup> N X N

vector is

about the measurement model

ð t

ð t

0

ð Þ λ1; λ1; λ1; λ<sup>1</sup>

ð Þ λ1; λ1; λ1; λ<sup>1</sup>

σ2 <sup>θ</sup> <sup>¼</sup> <sup>1</sup> N X N

as well as the robot's yaw statistical measurement model

i¼1

R<sup>k</sup> ¼

<sup>W</sup><sup>k</sup> <sup>¼</sup> <sup>∂</sup><sup>f</sup> ∂w<sup>k</sup> ¼

Furthermore, the measurement noise covariance matrix is

ð t

i¼1

i¼1

By substituting an averaged Cartesian displacement, one considers n key-point observations to calculate the recursive noisy state vector xk.The Jacobian matrix J<sup>k</sup> of the robot's temporal matrix state transition <sup>A</sup><sup>k</sup> � <sup>x</sup>k, where xk <sup>¼</sup> xk�<sup>1</sup> <sup>þ</sup> <sup>v</sup> cos ð Þ <sup>θ</sup> <sup>Δ</sup>t, yk <sup>¼</sup> yk�<sup>1</sup> <sup>þ</sup> <sup>v</sup> cos ð Þ <sup>θ</sup> <sup>Δ</sup>t, and <sup>θ</sup><sup>k</sup> <sup>¼</sup> θ<sup>k</sup>�<sup>1</sup> þ ωΔt is stated by

<sup>J</sup> <sup>¼</sup> <sup>∂</sup>A<sup>k</sup> � <sup>x</sup><sup>k</sup> ∂x<sup>k</sup>�<sup>1</sup> ¼ ∂x ∂x ∂x ∂y ∂x ∂θ ∂x ∂v ∂x ∂ω ∂y ∂x ∂y ∂y ∂y ∂θ ∂y ∂v ∂y ∂ω ∂θ ∂x ∂θ ∂y ∂θ ∂θ ∂θ ∂v ∂θ ∂ω ∂v ∂x ∂v ∂y ∂v ∂θ ∂v ∂v ∂v ∂ω ∂ω ∂x ∂ω ∂y ∂ω ∂θ ∂ω ∂v ∂ω ∂ω 0 BBBBBBBBBBBBBBBB@ 1 CCCCCCCCCCCCCCCCA ¼ 1 0 �v sin ð Þ θ Δt cos ð Þ θ Δt 0 0 1 v cos ð Þ θ Δt sin ð Þ θ Δt 0 00 1 0 t 00 0 1 0 00 0 0 1 0 BBBBBB@ 1 CCCCCCA : (67)

Thus, a measurement is a 3D point arising from either divergent pair Eq. (10) or (11) and deployed by Proposition 1.1. Thus, the robot's measurement vector model z includes noise measurements. The Jacobian matrix H of the expected state model w.r.t. measurements is defined by

$$\mathbf{H} = \frac{\partial \mathbf{h}}{\partial \mathbf{x}} = \begin{pmatrix} \frac{\partial \Delta \mathbf{x}}{\partial \mathbf{x}\_{k-1}} & \frac{\partial \Delta \mathbf{x}}{\partial y\_{k-1}} & \frac{\partial \Delta \mathbf{x}}{\partial \theta\_{k-1}} \\ \frac{\partial \Delta y}{\partial \mathbf{x}\_{k-1}} & \frac{\partial \Delta y}{\partial y\_{k-1}} & \frac{\partial \Delta y}{\partial \theta\_{k-1}} \\ \frac{\partial \Delta \theta}{\partial \mathbf{x}\_{k-1}} & \frac{\partial \Delta \theta}{\partial y\_{k-1}} & \frac{\partial \Delta \theta}{\partial \theta\_{k-1}} \end{pmatrix}. \tag{68}$$

The process noise covariance matrix Q<sup>k</sup> is defined by

$$\mathbf{Q}\_k |2\pi\mathbf{E}|^{-1/2} \exp\left\{ 0.5(\mathbf{z} - \boldsymbol{\mu})^\mathsf{T} \boldsymbol{\Sigma}^{-1} (\mathbf{z} - \boldsymbol{\mu}) \right\}. \tag{69}$$

Let us define the nonstationary covariance matrix P,

$$\mathbf{P} = \begin{pmatrix} \sigma\_x^2 & 0 & 0 \\ 0 & \sigma\_y^2 & 0 \\ 0 & 0 & \sigma\_\theta^2 \end{pmatrix} \tag{70}$$

the matrix diagonal variances are experimental measurements that describe the trend of the robot motion's error.

The robot's motion covariance matrix was obtained experimentally through 500 tests—straight motion, right turns, left turns, clockwise and counterclockwise rotations, with N ¼ 100 tests for each type of motion. Exteroceptive sensing devices onboard are tied to the robot's geometry of motion, and with their observations the robot's posture can be estimated and therefore matched with the robot's deterministic kinematic model. From Section 3, the inverse (38) and direct (33) models were used experimentally to obtain the following statistical covariance about the measurement model

$$\sigma\_x^2 = \frac{1}{N} \sum\_{i=1}^N \left( \int\_t (\lambda\_1, \lambda\_1, \lambda\_1, \lambda\_1)^\top \cdot \dot{\mathbf{d}} \, \mathrm{d}t \cdot \cos\left( \int\_t \left( \frac{rW}{l\_1^2} \, \frac{rW}{l\_2^2} \, \frac{-rW}{l\_3^2} \, \frac{-rW}{l\_4^2} \right)^\top \cdot \dot{\mathbf{d}} \, \mathrm{d}t \right) - \overline{\mathbf{x}} \right)^2 \tag{71}$$

and

Δy ¼ ð Þ Δs sin θ<sup>k</sup>�<sup>1</sup> þ λ þ

Δθ ¼ λ þ

∂x ∂ω 1

CCCCCCCCCCCCCCCCA ¼

∂y ∂ω

∂θ ∂ω

∂v ∂ω

∂ω ∂ω

as well as Δθ is given by

190 Applications of Mobile Robots

θ<sup>k</sup>�<sup>1</sup> þ ωΔt is stated by

<sup>J</sup> <sup>¼</sup> <sup>∂</sup>A<sup>k</sup> � <sup>x</sup><sup>k</sup> ∂x<sup>k</sup>�<sup>1</sup>

defined by

¼

BBBBBBBBBBBBBBBB@

∂x ∂x

0

∂y ∂x

∂θ ∂x

∂v ∂x

∂ω ∂x

∂x ∂y

∂y ∂y

∂θ ∂y

∂v ∂y

∂ω ∂y

∂x ∂θ

∂y ∂θ

∂θ ∂θ

∂v ∂θ

∂ω ∂θ

<sup>H</sup> <sup>¼</sup> <sup>∂</sup><sup>h</sup>

<sup>Q</sup>k\_j j <sup>2</sup>π<sup>Σ</sup> �1=<sup>2</sup>

The process noise covariance matrix Q<sup>k</sup> is defined by

Let us define the nonstationary covariance matrix P,

∂x ¼¼

P ¼

∂x ∂v

∂y ∂v

∂θ ∂v

∂v ∂v

∂ω ∂v

π <sup>2</sup> � <sup>θ</sup><sup>a</sup> t�1

, (65)

1

CCCCCCA : (67)

: (68)

: (69)

CA, (70)

: (66)

1 0 �v sin ð Þ θ Δt cos ð Þ θ Δt 0 0 1 v cos ð Þ θ Δt sin ð Þ θ Δt 0 00 1 0 t 00 0 1 0 00 0 0 1

� � � �

π <sup>2</sup> � <sup>θ</sup><sup>a</sup> t�1 � �

By substituting an averaged Cartesian displacement, one considers n key-point observations to calculate the recursive noisy state vector xk.The Jacobian matrix J<sup>k</sup> of the robot's temporal matrix state transition <sup>A</sup><sup>k</sup> � <sup>x</sup>k, where xk <sup>¼</sup> xk�<sup>1</sup> <sup>þ</sup> <sup>v</sup> cos ð Þ <sup>θ</sup> <sup>Δ</sup>t, yk <sup>¼</sup> yk�<sup>1</sup> <sup>þ</sup> <sup>v</sup> cos ð Þ <sup>θ</sup> <sup>Δ</sup>t, and <sup>θ</sup><sup>k</sup> <sup>¼</sup>

0

BBBBBB@

Thus, a measurement is a 3D point arising from either divergent pair Eq. (10) or (11) and deployed by Proposition 1.1. Thus, the robot's measurement vector model z includes noise measurements. The Jacobian matrix H of the expected state model w.r.t. measurements is

> ∂Δx ∂xk�<sup>1</sup>

0

BBBBBBB@

∂Δy ∂xk�<sup>1</sup>

∂Δθ ∂xk�<sup>1</sup>

exp 0:5ð Þ <sup>z</sup> � <sup>μ</sup> <sup>⊤</sup>Σ�<sup>1</sup>

<sup>x</sup> 0 0 0 σ<sup>2</sup> <sup>y</sup> 0 0 0 σ<sup>2</sup>

σ2

0 B@

∂Δx <sup>∂</sup>yk�<sup>1</sup>

∂Δy <sup>∂</sup>yk�<sup>1</sup>

∂Δθ <sup>∂</sup>yk�<sup>1</sup>

n o

θ

1

∂Δx ∂θ<sup>k</sup>�<sup>1</sup> 1

CCCCCCCA

∂Δy ∂θ<sup>k</sup>�<sup>1</sup>

∂Δθ ∂θ<sup>k</sup>�<sup>1</sup>

ð Þ z � μ

$$\sigma\_y^2 = \frac{1}{N} \sum\_{i=1}^N \left( \int\_t (\lambda\_1, \lambda\_1, \lambda\_1, \lambda\_1)^\top \cdot \dot{\mathbf{\mathcal{Q}}} \, d\mathbf{t} \cdot \sin\left( \int\_t \left( \frac{rW}{l\_1^2} \, \frac{rW}{l\_2^2} \, \frac{-rW}{l\_3^2} \, \frac{-rW}{l\_4^2} \right)^\top \cdot \dot{\mathbf{\mathcal{Q}}} \, d\mathbf{t} \, d\mathbf{t} \right) - \overline{y} \right)^2,\tag{72}$$

as well as the robot's yaw statistical measurement model

$$
\sigma\_{\theta}^{2} = \frac{1}{N} \sum\_{i=1}^{N} \left( \int\_{t} \left( \frac{rW}{l\_{1}^{2}} \frac{rW}{l\_{2}^{2}} \frac{-rW}{l\_{3}^{2}} \frac{-rW}{l\_{4}^{2}} \right)^{\top} \cdot \dot{\mathbf{d}} \, \mathbf{d}t - \overline{\theta} \right)^{2} . \tag{73}
$$

Furthermore, the measurement noise covariance matrix is

$$\mathbf{R}\_{k} = \begin{pmatrix} \sigma\_{x\_{p}}^{2} & 0 & 0 & 0 & 0 & 0\\ 0 & \sigma\_{y\_{p}}^{2} & 0 & 0 & 0 & 0\\ 0 & 0 & \sigma\_{y\_{p}}^{2} & 0 & 0 & 0 \end{pmatrix},\tag{74}$$

and the matrix W<sup>k</sup> which is the partial derivative of the process model w.r.t. the process noise vector is

$$\mathbf{W}\_{k} = \frac{\partial f}{\partial \mathbf{w}\_{k}} = \begin{pmatrix} \frac{\partial \mathbf{x}\_{k}}{\partial w\_{x}} & \frac{\partial \mathbf{x}\_{k}}{\partial w\_{y}} & \frac{\partial \mathbf{x}\_{k}}{\partial w\_{\theta}} \\ \frac{\partial y\_{k}}{\partial w\_{x}} & \frac{\partial y\_{k}}{\partial w\_{y}} & \frac{\partial y\_{k}}{\partial w\_{\theta}} \\ \frac{\partial \theta\_{k}}{\partial w\_{x}} & \frac{\partial \theta\_{k}}{\partial w\_{y}} & \frac{\partial \theta\_{k}}{\partial w\_{\theta}} \end{pmatrix}. \tag{75}$$

The matrix V<sup>k</sup> ¼ ∂h=∂v<sup>k</sup> is the partial derivative w.r.t. the measurement noise vector,

$$\mathbf{V}\_{k} = \begin{pmatrix} \frac{\partial \mathbf{x}\_{k}}{\partial \upsilon\_{X\_{p}}} & \frac{\partial \mathbf{x}\_{k}}{\partial \upsilon\_{Y\_{p}}} & \frac{\partial \mathbf{x}\_{k}}{\partial \upsilon\_{Z\_{p}}}\\ \frac{\partial y\_{k}}{\partial \upsilon\_{X\_{p}}} & \frac{\partial y\_{k}}{\partial \upsilon\_{Y\_{p}}} & \frac{\partial y\_{k}}{\partial \upsilon\_{Z\_{p}}} \end{pmatrix}. \tag{76}$$

Therefore, the observation vector with Gauss noise w is

4.1.3. Update estimate

The update estimate is obtained by

4.1.4. Update error covariance

4.1.5. Deterministic control model

This step converges until uref

B being a control transition matrix.

It follows that state prediction is

prediction model is

4.1.6. State prediction

from the inverse kinematics equation, Eq. (38)

where, in the previous expression, u<sup>R</sup> ¼ s; θ; xR; yR

is the vector of the wheels' instantaneous measurements

<sup>R</sup> � u^<sup>t</sup>

and the error dispersion covariance matrix is also predicted at t þ 1

z<sup>k</sup> ¼ H � x<sup>k</sup> þ

Therefore, the prediction is firstly obtained through the robot's inverse position control model,

� �<sup>⊤</sup>

x<sup>k</sup> ¼ x<sup>k</sup>�<sup>1</sup> þ B � u<sup>t</sup>þ<sup>1</sup>

<sup>t</sup> � <sup>u</sup>ref

<sup>t</sup> � <sup>Ω</sup><sup>t</sup>þ<sup>1</sup> � <sup>Ω</sup>^ <sup>t</sup> � �:

<sup>R</sup> � x^<sup>k</sup> � �,

. Thus, <sup>Ω</sup>^ <sup>t</sup> <sup>¼</sup> <sup>2</sup><sup>π</sup>

� � <sup>&</sup>lt; <sup>ε</sup>u, where <sup>ε</sup><sup>u</sup> is the convergence error. Then, the robot'<sup>s</sup>

x<sup>k</sup>þ<sup>1</sup> ¼ Φ<sup>k</sup> þ x<sup>k</sup> þ q<sup>k</sup> (82)

<sup>R</sup> Δη1; <sup>2</sup><sup>π</sup>

<sup>R</sup> Δη2; <sup>2</sup><sup>π</sup>

� �<sup>⊤</sup>

<sup>R</sup> Δη3; <sup>2</sup><sup>π</sup>

<sup>R</sup> Δη<sup>4</sup>

<sup>Ω</sup><sup>t</sup>þ<sup>1</sup> <sup>¼</sup> <sup>Ω</sup><sup>t</sup> <sup>þ</sup> <sup>Λ</sup>�<sup>1</sup>

<sup>u</sup><sup>t</sup>þ<sup>1</sup> <sup>¼</sup> <sup>u</sup><sup>t</sup> � <sup>Λ</sup>�<sup>1</sup>

x^<sup>k</sup> ¼ x^�

The covariance matrix error dispersion of the system is updated

wx wy w<sup>θ</sup> 1

CA: (79)

http://dx.doi.org/10.5772/intechopen.79130

193

<sup>k</sup>�<sup>1</sup> <sup>þ</sup> <sup>K</sup>kð Þ <sup>z</sup><sup>k</sup> � Hx<sup>k</sup>�<sup>1</sup> : (80)

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

<sup>P</sup>^<sup>k</sup> <sup>¼</sup> <sup>P</sup><sup>k</sup> � <sup>K</sup>kHkPk: (81)

0 B@

Let us summarize the 3D points pAB and pBC obtained by Theorem 1.

#### 4.1. State feedback position control

This section describes in six general steps the combined use of the visual observers and the EKF geometric odometer as a recursive feedback for the robot's positioning control. The robot's deterministic kinematic model conveys predictions about the robot's geometry of motion and its observations. Therefore, the deterministic model is used to infer the robot's motion observations implicitly by the trinocular sensor. The following formulation illustrates how the EKF and the visual odometry model are fed back for the 4WD kinematics.

#### 4.1.1. Kalman gain

The initial estimate of Kalman gain is

$$\mathbf{k}\_k = \mathbf{P}\_k \mathbf{H}\_k^T \left(\mathbf{H}\_k \mathbf{P}\_k \mathbf{H}\_k^T + \mathbf{V}\_k \mathbf{R}\_x \mathbf{V}\_k^T\right)^{-1} \tag{77}$$

#### 4.1.2. Observation

From Proposition 1.1, the visual observers provide m 3D key-points from Theorem 1, pAB,BC

$$\Delta \mathbf{s}(\mathbf{p}) = \sqrt[2]{\|\mathbf{p}\_{t-1}\| + \|\mathbf{p}\_t\| - 2\|\mathbf{p}\_{t-1}\| \|\mathbf{p}\_t\| \cos\left(\beta\_{t-1} - \beta\_t\right)}$$

The angle of each key-point p or q w.r.t. to the robot in actual time is

$$\lambda\_t = \arcsin\left(\frac{\|\mathbf{p}\_t \sin\left(\hat{\beta}\right)\|}{\Delta s}\right).$$

and the local angle of the robot w.r.t. the robot's previous position is

$$
\phi = \lambda\_t + \left(\frac{\pi}{2} - \theta\_{t-1}\right)\_t
$$

thus the inferred displacement is

$$\mathbf{x}\_{k} = \begin{pmatrix} \frac{1}{m+n} \sum\_{i} \Delta s\_{i}^{a}(\mathbf{p}) \cos \left(\phi\_{i}^{a}\right) \\\frac{1}{m+n} \sum\_{i} \Delta s\_{i}^{a}(\mathbf{p}) \sin \left(\phi\_{i}^{a}\right) \\\lambda + \left(\frac{\pi}{2} - \Theta\_{t-1}^{a}\right) \end{pmatrix} . \tag{78}$$

Therefore, the observation vector with Gauss noise w is

$$\mathbf{z}\_k = \mathbf{H} \cdot \mathbf{x}\_k + \begin{pmatrix} w\_x \\ w\_y \\ w\_\theta \end{pmatrix}. \tag{79}$$

#### 4.1.3. Update estimate

V<sup>k</sup> ¼

Let us summarize the 3D points pAB and pBC obtained by Theorem 1.

4.1. State feedback position control

The initial estimate of Kalman gain is

thus the inferred displacement is

Δsð Þ¼ p

q

4.1.1. Kalman gain

192 Applications of Mobile Robots

4.1.2. Observation

∂xk ∂vXp

0

BBB@

∂yk ∂vXp

∂xk ∂vYp

∂yk ∂vYp

This section describes in six general steps the combined use of the visual observers and the EKF geometric odometer as a recursive feedback for the robot's positioning control. The robot's deterministic kinematic model conveys predictions about the robot's geometry of motion and its observations. Therefore, the deterministic model is used to infer the robot's motion observations implicitly by the trinocular sensor. The following formulation illustrates

how the EKF and the visual odometry model are fed back for the 4WD kinematics.

<sup>k</sup> HkPkH<sup>T</sup>

From Proposition 1.1, the visual observers provide m 3D key-points from Theorem 1, pAB,BC

<sup>λ</sup><sup>t</sup> <sup>¼</sup> arcsin <sup>∥</sup>p<sup>t</sup> sin <sup>β</sup>^

π <sup>2</sup> � <sup>θ</sup><sup>t</sup>�<sup>1</sup> � �

ϕ ¼ λ<sup>t</sup> þ

1 m þ n

0

BBBBB@

1 m þ n X i Δs a

X i Δs a

> π <sup>2</sup> � <sup>θ</sup><sup>a</sup> t�1 � �

λ þ

<sup>k</sup> <sup>þ</sup> <sup>V</sup>kRxV<sup>T</sup>

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

� �∥

,

<sup>i</sup>ð Þ <sup>p</sup> cos <sup>ϕ</sup><sup>a</sup>

<sup>i</sup>ð Þ <sup>p</sup> sin <sup>ϕ</sup><sup>a</sup>

i � � 1

CCCCCA

: (78)

i � �

,

� � <sup>2</sup>

Δs !

<sup>∥</sup> � <sup>2</sup>∥p<sup>t</sup>�<sup>1</sup>∥∥p<sup>t</sup>

k

� ��<sup>1</sup> (77)

<sup>∥</sup> cos <sup>β</sup><sup>t</sup>�<sup>1</sup> � <sup>β</sup><sup>t</sup>

<sup>k</sup><sup>k</sup> <sup>¼</sup> <sup>P</sup>kH<sup>⊤</sup>

<sup>∥</sup>p<sup>t</sup>�<sup>1</sup><sup>∥</sup> <sup>þ</sup> <sup>∥</sup>p<sup>t</sup>

The angle of each key-point p or q w.r.t. to the robot in actual time is

and the local angle of the robot w.r.t. the robot's previous position is

x<sup>k</sup> ¼

∂xk ∂vZp 1

CCCA: (76)

∂yk ∂vZp

The update estimate is obtained by

$$
\hat{\mathbf{x}}\_{k} = \hat{\mathbf{x}}\_{k-1}^{-} + \mathbf{K}\_{k} (\mathbf{z}\_{k} - \mathbf{H}\mathbf{x}\_{k-1}).\tag{80}
$$

#### 4.1.4. Update error covariance

The covariance matrix error dispersion of the system is updated

$$
\ddot{\mathbf{P}}\_k = \mathbf{P}\_k - \mathbf{K}\_k \mathbf{H}\_k \mathbf{P}\_k. \tag{81}
$$

#### 4.1.5. Deterministic control model

Therefore, the prediction is firstly obtained through the robot's inverse position control model, from the inverse kinematics equation, Eq. (38)

$$\mathbf{\Omega}\_{t+1} = \mathbf{\Omega}\_t + \mathbf{\Lambda}\_t^{-1} \cdot \left(\mathbf{u}\_{\mathbb{R}}^{\text{ref}} - \hat{\mathbf{x}}\_k\right)\_{\prime 1}$$

where, in the previous expression, u<sup>R</sup> ¼ s; θ; xR; yR � �<sup>⊤</sup> . Thus, <sup>Ω</sup>^ <sup>t</sup> <sup>¼</sup> <sup>2</sup><sup>π</sup> <sup>R</sup> Δη1; <sup>2</sup><sup>π</sup> <sup>R</sup> Δη2; <sup>2</sup><sup>π</sup> <sup>R</sup> Δη3; <sup>2</sup><sup>π</sup> <sup>R</sup> Δη<sup>4</sup> � �<sup>⊤</sup> is the vector of the wheels' instantaneous measurements

$$\mathbf{u}\_{t+1} = \mathbf{u}\_t - \boldsymbol{\Lambda}\_t^{-1} \cdot \left(\boldsymbol{\Omega}\_{t+1} - \hat{\boldsymbol{\Omega}}\_t\right).$$

This step converges until uref <sup>R</sup> � u^<sup>t</sup> � � <sup>&</sup>lt; <sup>ε</sup>u, where <sup>ε</sup><sup>u</sup> is the convergence error. Then, the robot'<sup>s</sup> prediction model is

$$
\overline{\mathbf{x}}\_{k} = \overline{\mathbf{x}}\_{k-1} + \mathbf{B} \cdot \mathbf{u}\_{t+1},
$$

B being a control transition matrix.

#### 4.1.6. State prediction

It follows that state prediction is

$$
\overline{\mathbf{x}}\_{k+1} = \mathbf{0}\_k + \overline{\mathbf{x}}\_k + \mathbf{q}\_k \tag{82}
$$

and the error dispersion covariance matrix is also predicted at t þ 1

$$\mathbf{P}\_{k+1} = \mathbf{P}\_k + \left(\mathbf{A} + \mathbf{A}^\top\right) \mathbf{P}\_k \Delta t + \left(\mathbf{A} \mathbf{P}\_k \mathbf{A}^\top - \Sigma\_W\right) \Delta t^2 \tag{83}$$

Figure 4b shows the dead-reckoning and the EKF Cartesian absolute errors, taken as main reference for the visual global reference system. As for the direct dead-reckoning measurements, the absolute error grows exponentially, where the position observation starts diverging before the robot reaches the third turn. As for the EKF model, the Cartesian error w.r.t. the

Figure 5. Errors' convergence behavior. (a) EKF variances over time. (b) Key-point map's divergence using state vectors.

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

http://dx.doi.org/10.5772/intechopen.79130

195

As for Figure 4c and d, the EKF's Cartesian and angular absolute errors w.r.t. the global visual tracker are shown. In Figure 4d, the local minimums and maximums determine the Cartesian

Finally, Figure 5a shows the covariance error behavior obtained at each control loop during the EKF recursive calculations. Figure 5b is a mapping of the measured key-points registered

This chapter presented a visual odometry scheme for a trinocular divergent visual system that was combined with an EKF for visual odometry estimation. The proposed trinocular geometric model observer geometrically combined adjacent radial views. About 20% of adjacent multiview overlapping data allowed inference of small volumes of depth information. In measuring 3D key-points, the X-axis metrical error was reported to be lower than 7 cm, with error less

using the state vector (posture) of a robot's turn to illustrate the map's divergence.

global reference does not diverge but preserves bounded error magnitudes.

regions where the robot performed its turns.

5. Conclusion

From the previous step, the estimation process repeats again, going to step one. The previous Kalman process is performed until the robot reaches the goal and the estimation error converges by numerical approximation according to ð Þ x<sup>k</sup> � x^<sup>k</sup> ≤ εx.

Therefore, Figure 4a shows the robot's trajectory obtained by the different comparative approaches conducted in this study. The postures measured by an external visual global reference system are the main references to be compared with. The EKF estimation was obtained by the use of Theorem 1, Proposition 1.1, and Eqs. (71)–(77). In addition, the trinocular key-points used as inputs of the visual odometry model inferred the robot's displacements, which are shown in same Figure 4a. Furthermore, the dead-reckoning robot system was deployed to infer the robot's postures and is also shown in Figure 4a. Raw odometry refers to the robot's dead-reckoning kinematic model used as a mean for direct posture observation through direct kinematics (33) and inverse kinematics (38), but using direct encoder readings by (14).

Figure 4. Positions and errors. (a) Cartesian positions. (b) Errors vs. global reference. (c) EKF's Cartesian errors' convergence. (d) EKF's angle error convergence.

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry http://dx.doi.org/10.5772/intechopen.79130 195

Figure 5. Errors' convergence behavior. (a) EKF variances over time. (b) Key-point map's divergence using state vectors.

Figure 4b shows the dead-reckoning and the EKF Cartesian absolute errors, taken as main reference for the visual global reference system. As for the direct dead-reckoning measurements, the absolute error grows exponentially, where the position observation starts diverging before the robot reaches the third turn. As for the EKF model, the Cartesian error w.r.t. the global reference does not diverge but preserves bounded error magnitudes.

As for Figure 4c and d, the EKF's Cartesian and angular absolute errors w.r.t. the global visual tracker are shown. In Figure 4d, the local minimums and maximums determine the Cartesian regions where the robot performed its turns.

Finally, Figure 5a shows the covariance error behavior obtained at each control loop during the EKF recursive calculations. Figure 5b is a mapping of the measured key-points registered using the state vector (posture) of a robot's turn to illustrate the map's divergence.

#### 5. Conclusion

<sup>P</sup><sup>k</sup>þ<sup>1</sup> <sup>¼</sup> <sup>P</sup><sup>k</sup> <sup>þ</sup> <sup>A</sup> <sup>þ</sup> <sup>A</sup><sup>⊤</sup> <sup>P</sup>kΔ<sup>t</sup> <sup>þ</sup> APkA<sup>⊤</sup> � <sup>Σ</sup><sup>W</sup>

verges by numerical approximation according to ð Þ x<sup>k</sup> � x^<sup>k</sup> ≤ εx.

encoder readings by (14).

194 Applications of Mobile Robots

gence. (d) EKF's angle error convergence.

From the previous step, the estimation process repeats again, going to step one. The previous Kalman process is performed until the robot reaches the goal and the estimation error con-

Therefore, Figure 4a shows the robot's trajectory obtained by the different comparative approaches conducted in this study. The postures measured by an external visual global reference system are the main references to be compared with. The EKF estimation was obtained by the use of Theorem 1, Proposition 1.1, and Eqs. (71)–(77). In addition, the trinocular key-points used as inputs of the visual odometry model inferred the robot's displacements, which are shown in same Figure 4a. Furthermore, the dead-reckoning robot system was deployed to infer the robot's postures and is also shown in Figure 4a. Raw odometry refers to the robot's dead-reckoning kinematic model used as a mean for direct posture observation through direct kinematics (33) and inverse kinematics (38), but using direct

Figure 4. Positions and errors. (a) Cartesian positions. (b) Errors vs. global reference. (c) EKF's Cartesian errors' conver-

Δt

<sup>2</sup> (83)

This chapter presented a visual odometry scheme for a trinocular divergent visual system that was combined with an EKF for visual odometry estimation. The proposed trinocular geometric model observer geometrically combined adjacent radial views. About 20% of adjacent multiview overlapping data allowed inference of small volumes of depth information. In measuring 3D key-points, the X-axis metrical error was reported to be lower than 7 cm, with error less than 10 cm in þY component and less than 3 cm in �Y (vertical). Likewise, we found an averaged Z-axis error less than 5 cm (depth). Such errors were mostly produced by the angular divergence of the object w.r.t. the central camera, rather than linear distances. Indoor experiments, measuring distances up to 10 m were developed. In addition, a set of experimental results in convergent robot's course gave closed loops, and as the robot moved, the trinocular sensor incrementally stored environmental 3D key-points.

[4] Jaeheon J, Correll N, Mulligan J. Trinocular visual odometry for divergent views with

4WD Robot Posture Estimation by Radial Multi-View Visual Odometry

http://dx.doi.org/10.5772/intechopen.79130

197

[5] Dellaert F, Kaess M. Probabilistic structure matching for visual slam with a multi-camera

[6] Lee C-H, Kwon S, Lee J-h, Lim Y-C, Lee M. Improvement of stereo vision-based position and velocity estimation and tracking using a stripe-based disparity estimation and inverse perspective map-based extended kalman filter. Optics and Lasers in Engineering. 2010;

[7] Harmat A, Wang DWL, Sharf I, Waslander SL, Tribou MJ. Multi-camera parallel tracking and mapping with non-overlapping fields of view. The International Journal of Robotics

[8] Seet G, Wei M, Han W. Efficient visual odometry estimation using stereo camera. In: 11th IEEE International Conference on Control and Automation; 2014. pp. 1399-1403

[9] Kanade T, Badino H, Yamamoto A. Visual odometry by multi-frame feature integration. In: IEEE International Conference on Computer Vision Workshops; December 2013. pp.

[10] Liang Z, Xu X. Improved visual odometry method for matching 3D point cloud data. In:

[11] Llorca DF, Parra I, Sotelo MA, Ocaña M. Robust visual odometry for vehicle localization in

[12] Rendon-Mancha JM, Fuentes-Pacheco J, Ruiz-Ascencio J. Visual simultaneous localization

[13] Neira J, Cisneros R, Lavion JE, Hernandez E, Ibarra JM. Visual slam with oriented landmarks and partial odometry. In: 21st International Conference on Electrical Communica-

[14] Hong S, Ye C. A pose graph based visual slam algorithm for robot pose estimation. In:

[15] Williams B, Reid I. On combining visual slam and visual odometry. In: IEEE International

[16] Huitl R, Schroth G, Kranz M, Steinbach E, Hilsenbeck S, Moller A. Scale-preserving longterm visual odometry for indoor navigation. In: International Conference on Indoor Posi-

[17] Martínez-García EA, Lerin E, Torres-Cordoba R. A multi-configuration kinematic model for active drive/steer four-wheel robot structures. Robotica. 2015;33:2309-2329. Cambridge Uni-

and mapping: A survey. Artificial Intelligence Review. 2015;43(1):5581

Conference on Robotics and Automation; May 2010. pp. 3494-3500

33rd Chinese Control Conference; July 2014; pp. 8474-8478

tions and Computers; February–March 2011. pp. 39-45

World Automation Congress; August 2014. pp. 917-922

tioning and Indoor Navigation; November 2012. p. 110

urban environments. Robotica. 2010;28:441-452

minimal overlap. In: IEEE Workshop on Robot Vision; 2013. pp. 229-236

rig. Computer Vision and Image Understanding. 2010;114(2):286-296

48(9):859-886

222-229

versity Press

Research. 2015;34(12):1480-1500

The robot's trajectory was obtained by different comparative approaches conducted in this study. The postures were measured by an external visual global reference system, which was the main reference system to be compared with. The robotic platform's kinematics was modeled in terms of a dead-reckoning approach. The direct and the inverse solutions were combined to produce a recursive linearized control model and this was used as the prediction model for EKF estimator. The dead-reckoning robot system was deployed to infer the robot's postures using directly the four encoders' readings, with good results obtained only for very short paths. As a comparative perspective, using only the 4WD dead-reckoning system the posture exponentially diverged.

We found bounded Cartesian error for this 4WD robot by deploying the EKF. The trinocular 3D key-points were used as inputs of the visual odometry model that inferred the robot's displacements by geometrical triangulations.

#### Author details

Edgar Alonso Martínez-García<sup>1</sup> \* and Luz Abril Torres-Méndez<sup>2</sup>

\*Address all correspondence to: edmartin@uacj.mx


#### References


[4] Jaeheon J, Correll N, Mulligan J. Trinocular visual odometry for divergent views with minimal overlap. In: IEEE Workshop on Robot Vision; 2013. pp. 229-236

than 10 cm in þY component and less than 3 cm in �Y (vertical). Likewise, we found an averaged Z-axis error less than 5 cm (depth). Such errors were mostly produced by the angular divergence of the object w.r.t. the central camera, rather than linear distances. Indoor experiments, measuring distances up to 10 m were developed. In addition, a set of experimental results in convergent robot's course gave closed loops, and as the robot moved, the trinocular

The robot's trajectory was obtained by different comparative approaches conducted in this study. The postures were measured by an external visual global reference system, which was the main reference system to be compared with. The robotic platform's kinematics was modeled in terms of a dead-reckoning approach. The direct and the inverse solutions were combined to produce a recursive linearized control model and this was used as the prediction model for EKF estimator. The dead-reckoning robot system was deployed to infer the robot's postures using directly the four encoders' readings, with good results obtained only for very short paths. As a comparative perspective, using only the 4WD dead-reckoning system the

We found bounded Cartesian error for this 4WD robot by deploying the EKF. The trinocular 3D key-points were used as inputs of the visual odometry model that inferred the robot's

\* and Luz Abril Torres-Méndez<sup>2</sup>

[1] Milella A, Reina G. Towards autonomous agriculture: Automatic ground detection using

[2] Gwo-Long L, Chi-Cheng C. Acquisition of translational motion by the parallel trinocular.

[3] Tae Choi B, Kim J-H, Jin Chung M. Recursive estimation of motion and a scene model with a two-camera system of divergent view. Pattern Recognition. 2010;43(6):2265-

1 Laboratorio de Robótica, Universidad Autónoma de Ciudad Juárez, Mexico

2 Robotics Active Vision Group, CINVESTAV, Campus Saltillo, Mexico

trinocular stereovision. Sensors. 2012;12(9):12405-12423

sensor incrementally stored environmental 3D key-points.

posture exponentially diverged.

Edgar Alonso Martínez-García<sup>1</sup>

Author details

196 Applications of Mobile Robots

References

2280

displacements by geometrical triangulations.

\*Address all correspondence to: edmartin@uacj.mx

Information Sciences. 2008;178:137-151


[18] Martinez-Garcia E, Mar A, Torres-Cordoba T. Dead-reckoning inverse and direct kinematic solution of a 4W independent driven rover. In: IEEE ANDESCON; 15–17 September 2010; Bogota, Colombia

**Chapter 10**

Provisional chapter

**IntelliSoC: A System Level Design and Conception of a**

DOI: 10.5772/intechopen.79265

This chapter presents a system level design and conception of a System-on-a-Chip (SoC) for the execution of cognitive agents. The computational architecture of this SoC will be presented using the cognitive model of the concurrent autonomous agent (CAA) as a reference. This cognitive model comprises three levels that run concurrently, namely the reactive level, the instinctive level and the cognitive level. The reactive level executes a fast perception-action cycle. The instinctive level receives perceptions from and sends the active behavior to the reactive level, and using a Knowledge Based System (KBS) executes plans by selecting reactive behaviors. The cognitive level receives symbolic information from the instinctive level to update its logical world model, used for planning and sends new local goals to instinctive level. Thus, this work proposes a novel SoC whose architecture fits the computational demands of the aforementioned cognitive model, allowing for

Every entity that can perceive its environment and perform actions upon it can be called an agent [1]. When this process is achieved using knowledge about the environment, then the agent is a cognitive agent [2]. Cognition, according to [3], is a process that allows to a system to robustly behave adaptively and autonomously, with anticipatory capabilities. The authors proceed by classifying cognitive systems in two broad classes, namely the cognitivist and the

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited.

© 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use,

distribution, and reproduction in any medium, provided the original work is properly cited.

IntelliSoC: A System Level Design and Conception of a

**System-on-a-Chip (SoC) to Cognitive Agents**

System-on-a-Chip (SoC) to Cognitive Agents

Diego Ferreira, Augusto Loureiro da Costa and

Diego Ferreira, Augusto Loureiro da Costa and

Additional information is available at the end of the chapter

fast, energy-efficient, embedded intelligent applications.

Keywords: cognitive agents, intelligent robots, mobile robots

Additional information is available at the end of the chapter

Wagner Luiz Alves De Oliveira

Wagner Luiz Alves De Oliveira

http://dx.doi.org/10.5772/intechopen.79265

**Architecture**

Abstract

1. Introduction

Architecture

[19] Martinez-Garcia EA, Torres-Cordoba T. 4WD skid-steer trajectory control of a rover with spring-based suspension analysis. In: International Conference in Intelligent Robotics and Applications (ICIRA2010); 10–12 November 2010; Shanghai, China

#### **IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture** IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

DOI: 10.5772/intechopen.79265

Diego Ferreira, Augusto Loureiro da Costa and Wagner Luiz Alves De Oliveira Diego Ferreira, Augusto Loureiro da Costa and Wagner Luiz Alves De Oliveira

Additional information is available at the end of the chapter Additional information is available at the end of the chapter

http://dx.doi.org/10.5772/intechopen.79265

#### Abstract

[18] Martinez-Garcia E, Mar A, Torres-Cordoba T. Dead-reckoning inverse and direct kinematic solution of a 4W independent driven rover. In: IEEE ANDESCON; 15–17 September

[19] Martinez-Garcia EA, Torres-Cordoba T. 4WD skid-steer trajectory control of a rover with spring-based suspension analysis. In: International Conference in Intelligent Robotics and

Applications (ICIRA2010); 10–12 November 2010; Shanghai, China

2010; Bogota, Colombia

198 Applications of Mobile Robots

This chapter presents a system level design and conception of a System-on-a-Chip (SoC) for the execution of cognitive agents. The computational architecture of this SoC will be presented using the cognitive model of the concurrent autonomous agent (CAA) as a reference. This cognitive model comprises three levels that run concurrently, namely the reactive level, the instinctive level and the cognitive level. The reactive level executes a fast perception-action cycle. The instinctive level receives perceptions from and sends the active behavior to the reactive level, and using a Knowledge Based System (KBS) executes plans by selecting reactive behaviors. The cognitive level receives symbolic information from the instinctive level to update its logical world model, used for planning and sends new local goals to instinctive level. Thus, this work proposes a novel SoC whose architecture fits the computational demands of the aforementioned cognitive model, allowing for fast, energy-efficient, embedded intelligent applications.

Keywords: cognitive agents, intelligent robots, mobile robots

#### 1. Introduction

Every entity that can perceive its environment and perform actions upon it can be called an agent [1]. When this process is achieved using knowledge about the environment, then the agent is a cognitive agent [2]. Cognition, according to [3], is a process that allows to a system to robustly behave adaptively and autonomously, with anticipatory capabilities. The authors proceed by classifying cognitive systems in two broad classes, namely the cognitivist and the

> © 2016 The Author(s). Licensee InTech. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and eproduction in any medium, provided the original work is properly cited. © 2019 The Author(s). Licensee IntechOpen. This chapter is distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

emergent systems. Inside the cognitivist class goes systems that relies on symbolic representation and information processing. In the second class, the emergent systems, are connectionist, dynamical and enactive systems.

representation language employed by the Street Engine is the Street Language, designed to map a variant of the Rete algorithm to the hardware. The street engine is controlled by events, where each producer stores a subset of the working memory (corresponding to the alpha memories in the Rete algorithm) and changes in the working memory of one producer causes changes in

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

http://dx.doi.org/10.5772/intechopen.79265

201

Deviating from production systems, a processor oriented to the real-time decision making of mobile robots is proposed by [14]. The processor comprises four search processors with 8 threads each for trajectory planning and a reinforcement learning acceleration module for obstacle avoidance. The search processors consist of a 6-stage pipeline that uses a three-level cache to implement a transposition and avoid redundant computation between threads. The reinforcement learning accelerator, in turn, uses a 2D array of processing elements to implement a SIMD architecture. A penalty is considered when a search processor tries to plan a

The common theme among these works is that they are concerned with expert (production) systems. According to [4], while cognitive architectures are intended to perform successfully in a broad range of domains, expert systems have a narrower range of applications. The authors then continues by saying that cognitive architectures "offers accounts of intelligent behaviors at the system level, rather than at the level of component methods designed for specialized tasks". Therefore, the design of a SoC for cognitive systems can accomplish the optimized performance of a dedicated low-level machine while maintaining its powerful intelligent

This chapter is divided as follows. In Section 2, the CAA is presented, with its levels explained. Section 3 then exposes how knowledge is represented and inference is performed by the CAA KBS (in both instinctive and cognitive levels). Sections 4 and 5 explains the Rete and Graphplan algorithms, laying the basis for the following sections, which describes the hardware architectures proposed to implement these algorithms. First, in Section 6, the overall architecture is described. Then, Section 7 describes the Rete RISC machine. And in Section 8 the cognitive module is presented. Section 9 shows the results of simulations and some final considerations

The architecture of the concurrent autonomous agent (CAA) was inspired by the generic model for cognitive agents. This model comprises three levels: the reactive level, the instinctive level and the cognitive level [15]. The CAA levels are shown in Figure 1 [5, 6]. As can be seen in this figure, the reactive level is responsible for interacting with the environment. It contains reactive behaviors that enables the agent to perform a fast perception-action cycle. The perceptions are sent to the instinctive level which, in turn, uses it to update the state of the world that it maintains in a KBS. It also uses this information to update the logical model of the world in the cognitive level and select the reactive behavior in the reactive level. Finally, the cognitive level does the planning, sending

local goals to the instinctive level, that will coordinate the execution of the plan [6].

other memories of producers affected by the change.

behavior, which justifies the importance of this work.

2. The concurrent autonomous agent (CAA)

trajectory that collides with an obstacle.

are presented in Section 10.

There are aspects of cognitive agents that remain invariant in time and over different tasks. These aspects generally include the short-term and long-term memories where knowledge is stored, the knowledge representation structure and the processes that performs over the previous elements (possibly changing its contents, like learning). The aspects cited above are comprised by a cognitive architecture [4].

An example of architecture for cognitive agents is the concurrent autonomous agent (CAA), an autonomous agent architecture for mobile robots that has already proven to be very powerful [5–7]. Three parallel levels compose this architecture: the reactive, the instinctive and the cognitive levels. The first is responsible for executing the perception-action cycle, the second uses a knowledge based system (KBS) to select behaviors in the reactive level, and the third also uses a KBS, but for planning.

In [8] the CAA was embedded in a microcontrollers network specially designed to fit its cognitive architecture. The intention of the authors was to optimize the performance of the agent for an embedded environment, allowing it to be physically embedded into a mobile robot. In this work a step forward is given in that direction, since its main objective is to design a System-on-a-Chip (SoC) dedicated to the execution of the CAA.

Hardware design for cognitivist systems (using [3] aforementioned classification) is not a recent concern. The first paper about the Rete matching algorithm [9] already presents a lowlevel (assembly) implementation of the matching algorithm for production systems. Years later, [10] designed a reduced instruction set computer (RISC) machine for OPS production systems, focusing on optimizing the branch prediction unit for the task. A more recent approach by [11] proposes a parallelization strategy to use the parallel processing power of graphics processing units (GPUs) for Rete pattern matching.

Still searching for dedicated hardware for the Rete algorithm, [12] present a special purpose machine that is connected to a host computer as a support for agents execution in software. The authors also shown that increasing the number of inputs of the beta nodes in the network generated by the algorithm enhances the performance of the proposed hardware: using four or five inputs in a join node of the beta network allowed for a 15–20% increase in performance. For the operation of the processor the network is compiled into an inner representation and loaded into an external RAM. A Token Queue Unit stores partial matches, controlled by a microprogram of the Control Unit and by a Match Unit.

The authors in [13] propose a processor for the execution of production systems aiming applications of real-time embedded general artificial intelligence. The production system executed by the processor is the Street system. The processor, named Street Engine, interprets RISC instructions from its ISA, which is composed by only one type of instructions: a form of parallel production rules. Hence, instead of being executed sequentially, the instructions are executed in parallel by hardware units named producers. The producers are interconnected in a network-on-a-chip (NoC), and each one has only one associated production. The knowledge representation language employed by the Street Engine is the Street Language, designed to map a variant of the Rete algorithm to the hardware. The street engine is controlled by events, where each producer stores a subset of the working memory (corresponding to the alpha memories in the Rete algorithm) and changes in the working memory of one producer causes changes in other memories of producers affected by the change.

Deviating from production systems, a processor oriented to the real-time decision making of mobile robots is proposed by [14]. The processor comprises four search processors with 8 threads each for trajectory planning and a reinforcement learning acceleration module for obstacle avoidance. The search processors consist of a 6-stage pipeline that uses a three-level cache to implement a transposition and avoid redundant computation between threads. The reinforcement learning accelerator, in turn, uses a 2D array of processing elements to implement a SIMD architecture. A penalty is considered when a search processor tries to plan a trajectory that collides with an obstacle.

The common theme among these works is that they are concerned with expert (production) systems. According to [4], while cognitive architectures are intended to perform successfully in a broad range of domains, expert systems have a narrower range of applications. The authors then continues by saying that cognitive architectures "offers accounts of intelligent behaviors at the system level, rather than at the level of component methods designed for specialized tasks". Therefore, the design of a SoC for cognitive systems can accomplish the optimized performance of a dedicated low-level machine while maintaining its powerful intelligent behavior, which justifies the importance of this work.

This chapter is divided as follows. In Section 2, the CAA is presented, with its levels explained. Section 3 then exposes how knowledge is represented and inference is performed by the CAA KBS (in both instinctive and cognitive levels). Sections 4 and 5 explains the Rete and Graphplan algorithms, laying the basis for the following sections, which describes the hardware architectures proposed to implement these algorithms. First, in Section 6, the overall architecture is described. Then, Section 7 describes the Rete RISC machine. And in Section 8 the cognitive module is presented. Section 9 shows the results of simulations and some final considerations are presented in Section 10.

#### 2. The concurrent autonomous agent (CAA)

emergent systems. Inside the cognitivist class goes systems that relies on symbolic representation and information processing. In the second class, the emergent systems, are connectionist,

There are aspects of cognitive agents that remain invariant in time and over different tasks. These aspects generally include the short-term and long-term memories where knowledge is stored, the knowledge representation structure and the processes that performs over the previous elements (possibly changing its contents, like learning). The aspects cited above are

An example of architecture for cognitive agents is the concurrent autonomous agent (CAA), an autonomous agent architecture for mobile robots that has already proven to be very powerful [5–7]. Three parallel levels compose this architecture: the reactive, the instinctive and the cognitive levels. The first is responsible for executing the perception-action cycle, the second uses a knowledge based system (KBS) to select behaviors in the reactive level, and the third

In [8] the CAA was embedded in a microcontrollers network specially designed to fit its cognitive architecture. The intention of the authors was to optimize the performance of the agent for an embedded environment, allowing it to be physically embedded into a mobile robot. In this work a step forward is given in that direction, since its main objective is to design

Hardware design for cognitivist systems (using [3] aforementioned classification) is not a recent concern. The first paper about the Rete matching algorithm [9] already presents a lowlevel (assembly) implementation of the matching algorithm for production systems. Years later, [10] designed a reduced instruction set computer (RISC) machine for OPS production systems, focusing on optimizing the branch prediction unit for the task. A more recent approach by [11] proposes a parallelization strategy to use the parallel processing power of

Still searching for dedicated hardware for the Rete algorithm, [12] present a special purpose machine that is connected to a host computer as a support for agents execution in software. The authors also shown that increasing the number of inputs of the beta nodes in the network generated by the algorithm enhances the performance of the proposed hardware: using four or five inputs in a join node of the beta network allowed for a 15–20% increase in performance. For the operation of the processor the network is compiled into an inner representation and loaded into an external RAM. A Token Queue Unit stores partial matches, controlled by a

The authors in [13] propose a processor for the execution of production systems aiming applications of real-time embedded general artificial intelligence. The production system executed by the processor is the Street system. The processor, named Street Engine, interprets RISC instructions from its ISA, which is composed by only one type of instructions: a form of parallel production rules. Hence, instead of being executed sequentially, the instructions are executed in parallel by hardware units named producers. The producers are interconnected in a network-on-a-chip (NoC), and each one has only one associated production. The knowledge

a System-on-a-Chip (SoC) dedicated to the execution of the CAA.

graphics processing units (GPUs) for Rete pattern matching.

microprogram of the Control Unit and by a Match Unit.

dynamical and enactive systems.

200 Applications of Mobile Robots

also uses a KBS, but for planning.

comprised by a cognitive architecture [4].

The architecture of the concurrent autonomous agent (CAA) was inspired by the generic model for cognitive agents. This model comprises three levels: the reactive level, the instinctive level and the cognitive level [15]. The CAA levels are shown in Figure 1 [5, 6]. As can be seen in this figure, the reactive level is responsible for interacting with the environment. It contains reactive behaviors that enables the agent to perform a fast perception-action cycle. The perceptions are sent to the instinctive level which, in turn, uses it to update the state of the world that it maintains in a KBS. It also uses this information to update the logical model of the world in the cognitive level and select the reactive behavior in the reactive level. Finally, the cognitive level does the planning, sending local goals to the instinctive level, that will coordinate the execution of the plan [6].

3. Knowledge-based systems (KBS)

model of the world in the cognitive level one.

is shown in Figure 2 [6].

Figure 2. Block diagram of a KBS.

The CAA uses a KBS in its two higher levels: the instinctive and the reactive. Its inner structure

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

http://dx.doi.org/10.5772/intechopen.79265

203

All knowledge of the agent is stored in the facts base. The elements of the facts base use the format logicð object attribute value Þ to represent elements in the world. The facts base contains the states of the agent and of the environment in the KBS of the instinctive level and the logical

The format above is used also to form the premises of the rules in the rules base. But in this case they are restrictions or specifications that the facts in the facts base must met in order to fire the rule and execute its consequence, which may contain instructions on how to modify its own facts base, or the facts base of another level (adding, removing or updating facts). Additionally, in the rules syntax the fields (object, attribute and value) of the logical elements may contain

The rules may also contain filters to further specify restrictions on the values of the fields in the premise elements. Filters have the format filterð operator parameter1 parameter2 Þ, where parameter1 and parameter2 are variables or symbols present in some premise elements and

The inference engine uses the Rete matching algorithm to search the rules base for rules that are enable to fire by the facts in the facts base without having to iterate through both bases. The enabled rules then form the conflict set, and the inference engine must decide which rules in this sets should be fired. Finally, the inference engine executes the consequence of the fired

variables, which are denoted by a symbol preceded by a interrogation (?) token.

operator tells how they must compare for the rule to be fired.

rules and restart the process. This is the forward chaining algorithm.

Figure 1. Concurrent autonomous agent architecture.

The level that interacts with the environment executing a fast-perception-action cycle is the reactive level. It consists of a collection of reactive behaviors that determines the interaction of the agent with the environment. Only one behavior can be active at a time, and the instinctive level makes the selection. The architecture used in [8, 16] consists of a kinematic position controller for the omnidirectional robot AxéBot. The reactive behaviors were implemented based on the embedded kinematic controller. The behaviors implemented were simple: there is one behavior for each cardinal direction, i.e., selecting the behavior corresponds to selecting the direction (relative to the orientation of the robot) in which one wishes the robot to move onto.

The instinctive level, as the reactive level, is identical to the one purposed in [8]: its reasoning mechanism consists of a KBS that executes a plan generated by the cognitive level, sending symbolic information about the environment to the latter. The plans are executed by coordinating behavior selection in the reactive level, which sends the perceptions to this level.

The cognitive level also uses a KBS as automatic reasoning method. Its facts base consists of a logical model of the world. The inference engine is multi-cycle, meaning that it keeps running independent of the update of the facts base by the instinctive level. This level does the planning, coordinating the instinctive level for the execution of the plans. It is not used in this work.

## 3. Knowledge-based systems (KBS)

The CAA uses a KBS in its two higher levels: the instinctive and the reactive. Its inner structure is shown in Figure 2 [6].

All knowledge of the agent is stored in the facts base. The elements of the facts base use the format logicð object attribute value Þ to represent elements in the world. The facts base contains the states of the agent and of the environment in the KBS of the instinctive level and the logical model of the world in the cognitive level one.

The format above is used also to form the premises of the rules in the rules base. But in this case they are restrictions or specifications that the facts in the facts base must met in order to fire the rule and execute its consequence, which may contain instructions on how to modify its own facts base, or the facts base of another level (adding, removing or updating facts). Additionally, in the rules syntax the fields (object, attribute and value) of the logical elements may contain variables, which are denoted by a symbol preceded by a interrogation (?) token.

The rules may also contain filters to further specify restrictions on the values of the fields in the premise elements. Filters have the format filterð operator parameter1 parameter2 Þ, where parameter1 and parameter2 are variables or symbols present in some premise elements and operator tells how they must compare for the rule to be fired.

The inference engine uses the Rete matching algorithm to search the rules base for rules that are enable to fire by the facts in the facts base without having to iterate through both bases. The enabled rules then form the conflict set, and the inference engine must decide which rules in this sets should be fired. Finally, the inference engine executes the consequence of the fired rules and restart the process. This is the forward chaining algorithm.

Figure 2. Block diagram of a KBS.

The level that interacts with the environment executing a fast-perception-action cycle is the reactive level. It consists of a collection of reactive behaviors that determines the interaction of the agent with the environment. Only one behavior can be active at a time, and the instinctive level makes the selection. The architecture used in [8, 16] consists of a kinematic position controller for the omnidirectional robot AxéBot. The reactive behaviors were implemented based on the embedded kinematic controller. The behaviors implemented were simple: there is one behavior for each cardinal direction, i.e., selecting the behavior corresponds to selecting the direction (relative to the orientation of the robot) in which one wishes the robot to move onto.

Figure 1. Concurrent autonomous agent architecture.

202 Applications of Mobile Robots

The instinctive level, as the reactive level, is identical to the one purposed in [8]: its reasoning mechanism consists of a KBS that executes a plan generated by the cognitive level, sending symbolic information about the environment to the latter. The plans are executed by coordinating behavior selection in the reactive level, which sends the perceptions to this level.

The cognitive level also uses a KBS as automatic reasoning method. Its facts base consists of a logical model of the world. The inference engine is multi-cycle, meaning that it keeps running independent of the update of the facts base by the instinctive level. This level does the planning, coordinating the instinctive level for the execution of the plans. It is not used in this work.

### 4. The Rete algorithm

The Rete algorithm is used in the inference engine of a KBS to efficiently match rules premises with the facts without the need of looping through both rules and facts bases at each inference cycle, greatly improving performance. It was proposed by Charles Forgy in 1979, in its doctoral thesis [17]. Its name, rete, is latin for network, because of how it organizes the information in the knowledge base.

• Σ ¼ ð Þ S; A; γ is the problem domain, with S being the set of states, A the set of actions and

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

An action a ∈ A is composed by two sets of propositions: prencond að Þ, its preconditions, and effectsð Þ¼ <sup>a</sup> effectsþð Þ<sup>a</sup> <sup>∪</sup> effects�ð Þ<sup>a</sup> , its effects, where effectsþð Þ<sup>a</sup> is the set of positive propositions and effects�ð Þa is the set of negative propositions in the effects of the action. For an action a to be applicable in a given state s, one should have precond að Þ ⊆s, and the new state would be given

Now back to Graphplan, given the action layer Aj and the propositional layer Pj�<sup>1</sup> preceding it, the former contains all actions a such that precond að Þ ⊆Pj�<sup>1</sup> and the later is composed by all

• connecting an action a∈ Aj to a proposition p∈ Pj�1, such that p ∈effectsþð Þa (positive arc);

• connecting an action a ∈ Aj to a proposition p∈ Pj�1, such that p ∈effects�ð Þa (negative arc).

<sup>¼</sup> <sup>Ø</sup>; otherwise they are said to be dependent.

If two actions are dependent, they are said to be mutually exclusive, or mutex for short. Another situation that makes two actions in the same layer to be mutex is when a precondition of one of them is mutex with a precondition of the other. And two propositions p and q are mutex if for all a<sup>1</sup> ∈ Aj such that p ∈effectsþð Þ a<sup>1</sup> is mutex with every a<sup>2</sup> ∈ Aj such that q∈effectsþð Þ a<sup>2</sup> , and

The algorithm works as follows. It first expands the planning graph, generating the direct layered graph described before, with the successive proposition and action layers. Then when a propositional layer Pj contains g, the expansion stops and the graph is searched backwards for a sequence of sets of non-mutex actions. The plan is then extracted from this sequence.

An important difference between the algorithm used here and the one from the standard approach is that here we use a KBS to represent knowledge and execute the inference. So the

expansion step The pseudo-code for the expansion step is given in Algorithm 1.

1: procedure EXPANDð Þ si ⊳si: i-th state layer

2: Aiþ<sup>1</sup> KBS:InferenceCycle si ð Þ ; A ⊳A: action profiles

<sup>¼</sup> <sup>Ø</sup> and

http://dx.doi.org/10.5772/intechopen.79265

205

Two actions a1, a<sup>2</sup> ∈ Aj are called independent if effects�ð Þ a<sup>1</sup> ∩ precond að Þ<sup>2</sup> ∪ effectsþð Þ a<sup>2</sup>

γ ¼ S � A ! S is a state transformation application;

propositions p such that p ∈Pj�1. Also, there are three types of edges:

• connecting a proposition p∈ Pj�<sup>1</sup> to an action a∈ Aj;

• sj ∈ S is the initial state; and

by γð Þ¼ s; a s � effects� ð Þ ð Þa ∪ effectsþð Þa .

effects�ð Þ a<sup>2</sup> ∩ precond að Þ<sup>1</sup> ∪ effectsþð Þ a<sup>1</sup>

=∃a ∈ Aj such that p, q∈ effectsþð Þa .

Algorithm 1 Planning graph expansion

3: siþ<sup>1</sup> ∪Aiþ<sup>1</sup>:effects<sup>þ</sup>

• g is the goal state.

and

The Rete algorithm starts by constructing a network of nodes and memories based on the premises of the rules and eventual filters they may contain. This network is divided in two parts: the alpha and the beta networks.

The alpha network is composed by the following nodes:


The beta network, in turn, have:


#### 5. The Graphplan algorithm

The cognitive level of the CAA contained a classical planner that used a KBS to perform a statespace search in order to generate the plan. This approach may produce an explosion in the number of states that, in turn, may overwhelm the memory capacity of the computational system that executes it. In the case of the present work, where a SoC is supposed to run the algorithm, this issue becomes specially expressive. The alternative considered here is the utilization of a planning graph algorithm search instead, which is known as Graphplan.

The Graphplan algorithm uses a propositional representation for states and actions, and its basic idea to reduce the search graph is to structure it as a sequence of layers comprising sets of states, or propositions. Following a state (proposition) layer there is an inclusive disjoint of actions, which is then followed by another propositional layer, and so on, alternately.

For a mathematical formulation of the Graphplan algorithm, one should first define mathematically a general planning problem. Let <sup>P</sup> <sup>¼</sup> <sup>Σ</sup>;sj; <sup>g</sup> be a planning problem, where:


4. The Rete algorithm

204 Applications of Mobile Robots

in the knowledge base.

parts: the alpha and the beta networks.

The beta network, in turn, have:

5. The Graphplan algorithm

The alpha network is composed by the following nodes:

• a Root Node, which is the entry point for new facts;

The Rete algorithm is used in the inference engine of a KBS to efficiently match rules premises with the facts without the need of looping through both rules and facts bases at each inference cycle, greatly improving performance. It was proposed by Charles Forgy in 1979, in its doctoral thesis [17]. Its name, rete, is latin for network, because of how it organizes the information

The Rete algorithm starts by constructing a network of nodes and memories based on the premises of the rules and eventual filters they may contain. This network is divided in two

• Constant Test Nodes (CTN), which checks whether the non-variable (constant) fields of

• Alpha Memories (AM), that stores facts that successfully passed through constant test nodes.

• Join Nodes (JN), where a set of test are performed to check variable binding consistency;

• Beta Memories (BM), which conjunctively "accumulates" facts that passed the corresponding

The cognitive level of the CAA contained a classical planner that used a KBS to perform a statespace search in order to generate the plan. This approach may produce an explosion in the number of states that, in turn, may overwhelm the memory capacity of the computational system that executes it. In the case of the present work, where a SoC is supposed to run the algorithm, this issue becomes specially expressive. The alternative considered here is the utilization of a planning graph algorithm search instead, which is known as Graphplan.

The Graphplan algorithm uses a propositional representation for states and actions, and its basic idea to reduce the search graph is to structure it as a sequence of layers comprising sets of states, or propositions. Following a state (proposition) layer there is an inclusive disjoint of

For a mathematical formulation of the Graphplan algorithm, one should first define mathemat-

actions, which is then followed by another propositional layer, and so on, alternately.

ically a general planning problem. Let <sup>P</sup> <sup>¼</sup> <sup>Σ</sup>;sj; <sup>g</sup> be a planning problem, where:

premises matches the corresponding ones in the current fact; and

JN tests in tokens, which are partial matches to specific premises; and

• Production Nodes, which are terminal nodes for full matches.

An action a ∈ A is composed by two sets of propositions: prencond að Þ, its preconditions, and effectsð Þ¼ <sup>a</sup> effectsþð Þ<sup>a</sup> <sup>∪</sup> effects�ð Þ<sup>a</sup> , its effects, where effectsþð Þ<sup>a</sup> is the set of positive propositions and effects�ð Þa is the set of negative propositions in the effects of the action. For an action a to be applicable in a given state s, one should have precond að Þ ⊆s, and the new state would be given by γð Þ¼ s; a s � effects� ð Þ ð Þa ∪ effectsþð Þa .

Now back to Graphplan, given the action layer Aj and the propositional layer Pj�<sup>1</sup> preceding it, the former contains all actions a such that precond að Þ ⊆Pj�<sup>1</sup> and the later is composed by all propositions p such that p ∈Pj�1. Also, there are three types of edges:


Two actions a1, a<sup>2</sup> ∈ Aj are called independent if effects�ð Þ a<sup>1</sup> ∩ precond að Þ<sup>2</sup> ∪ effectsþð Þ a<sup>2</sup> <sup>¼</sup> <sup>Ø</sup> and effects�ð Þ a<sup>2</sup> ∩ precond að Þ<sup>1</sup> ∪ effectsþð Þ a<sup>1</sup> <sup>¼</sup> <sup>Ø</sup>; otherwise they are said to be dependent.

If two actions are dependent, they are said to be mutually exclusive, or mutex for short. Another situation that makes two actions in the same layer to be mutex is when a precondition of one of them is mutex with a precondition of the other. And two propositions p and q are mutex if for all a<sup>1</sup> ∈ Aj such that p ∈effectsþð Þ a<sup>1</sup> is mutex with every a<sup>2</sup> ∈ Aj such that q∈effectsþð Þ a<sup>2</sup> , and =∃a ∈ Aj such that p, q∈ effectsþð Þa .

The algorithm works as follows. It first expands the planning graph, generating the direct layered graph described before, with the successive proposition and action layers. Then when a propositional layer Pj contains g, the expansion stops and the graph is searched backwards for a sequence of sets of non-mutex actions. The plan is then extracted from this sequence.

An important difference between the algorithm used here and the one from the standard approach is that here we use a KBS to represent knowledge and execute the inference. So the expansion step The pseudo-code for the expansion step is given in Algorithm 1.

Algorithm 1 Planning graph expansion

1: procedure EXPANDð Þ si ⊳si: i-th state layer 2: Aiþ<sup>1</sup> KBS:InferenceCycle si ð Þ ; A ⊳A: action profiles 3: siþ<sup>1</sup> ∪Aiþ<sup>1</sup>:effects<sup>þ</sup>

4: <sup>μ</sup>Aiþ<sup>1</sup> ð Þ <sup>a</sup>; <sup>b</sup> <sup>∈</sup> <sup>A</sup><sup>2</sup> <sup>i</sup>þ<sup>1</sup>, a 6¼ <sup>b</sup> <sup>∣</sup>Dependent að Þ ; <sup>b</sup> <sup>∨</sup> <sup>∃</sup>ð Þ <sup>p</sup>; <sup>q</sup> <sup>∈</sup>μsi : <sup>p</sup><sup>∈</sup> preconds að Þ, q∈ preconds bð Þg

$$\text{5:} \qquad \mu\\ \text{s}\_{i+1} \leftarrow \left\{ (p,q) \in s\_{i+1}^2, p \neq q \, | \forall (a,b) \in A\_{i+1}^2 : p \in \text{effect}^+(a) \land q \in \text{cf} \text{és} \, \text{s}^+(b) \to (a,b) \in \mu A\_{i+1} \right\}$$

3: return Ø

7: return π<sup>i</sup>

9: return Failure 10: end procedure

6. The proposed architecture

Figure 3. Block diagram of the proposed SoC.

Figure 3 shows an overview of the SoC architecture for cognitive agents proposed in this work. As mentioned before, the cognitive model of the CAA (Figure 1) was used as a base model for

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

http://dx.doi.org/10.5772/intechopen.79265

207

5: π<sup>i</sup> Search gð Þ ; Ø; i 6: if π<sup>i</sup> 6¼ Failure then

4: end if

8: end if

#### 6: end procedure

Whenever the set of WME in g - the goal state - is contained in a given state layer si, a recursive procedure must be executed to search for sets of non-mutex actions in each layer that could have produced all the WMEs in the goal state. This procedure is composed by the functions Search and Extract.

Algorithm 2 Search for non-mutex actions.

```
1: procedure SEARCHðg, πi, iÞ
2: if g ¼ Ø then
3: Π Extract ∪f g preconds að Þj∀a ∈πi ð Þ ; i � 1
4: if Π ¼ Failure then
5: return Failure
6: end if
7: return Π:πi
8: else
9: select any p ∈g
10: resolvers a∈ Ai jp∈ effectsþð Þa ∧ ∀b∈πi : ð Þ a; b ∉μAi

11: if resolvers ¼ Ø then
12: return Failure
13: end if
14: non-deterministically choose a ∈resolvers
15: return Search(g � effectsþð Þa , πi∪f ga , i)
16: end if
17: end procedure
```
#### Algorithm 3 Extract a plan.

#### 1: procedure EXTRACTð<sup>g</sup>, iÞ

2: if i ¼ 0 then

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture http://dx.doi.org/10.5772/intechopen.79265 207


10: end procedure

4: <sup>μ</sup>Aiþ<sup>1</sup> ð Þ <sup>a</sup>; <sup>b</sup> <sup>∈</sup> <sup>A</sup><sup>2</sup>

5: <sup>μ</sup>siþ<sup>1</sup> ð Þ <sup>p</sup>; <sup>q</sup> <sup>∈</sup>s<sup>2</sup>

6: end procedure

206 Applications of Mobile Robots

Search and Extract.

2: if g ¼ Ø then

6: end if

13: end if

16: end if

17: end procedure

2: if i ¼ 0 then

Algorithm 3 Extract a plan.

1: procedure EXTRACTð<sup>g</sup>, iÞ

8: else

7: return Π:π<sup>i</sup>

9: select any p ∈g

11: if resolvers ¼ Ø then 12: return Failure

1: procedure SEARCHð<sup>g</sup>, πi, iÞ

4: if Π ¼ Failure then 5: return Failure

preconds að Þ, q∈ preconds bð Þg

Algorithm 2 Search for non-mutex actions.

3: Π Extract ∪f g preconds að Þj∀a ∈π<sup>i</sup> ð Þ ; i � 1

10: resolvers a∈ Ai jp∈ effectsþð Þa ∧ ∀b∈π<sup>i</sup> : ð Þ a; b ∉μAi

14: non-deterministically choose a ∈resolvers

15: return Search(g � effectsþð Þa , πi∪f ga , i)

<sup>i</sup>þ<sup>1</sup>, a 6¼ <sup>b</sup> <sup>∣</sup>Dependent að Þ ; <sup>b</sup> <sup>∨</sup> <sup>∃</sup>ð Þ <sup>p</sup>; <sup>q</sup> <sup>∈</sup>μsi : <sup>p</sup><sup>∈</sup>

Whenever the set of WME in g - the goal state - is contained in a given state layer si, a recursive procedure must be executed to search for sets of non-mutex actions in each layer that could have produced all the WMEs in the goal state. This procedure is composed by the functions

<sup>i</sup>þ<sup>1</sup> : <sup>p</sup><sup>∈</sup> effectsþð Þ<sup>a</sup> <sup>∧</sup> <sup>q</sup><sup>∈</sup> effectsþð Þ!<sup>b</sup> ð Þ <sup>a</sup>; <sup>b</sup> <sup>∈</sup> <sup>μ</sup>Aiþ<sup>1</sup>

<sup>i</sup>þ<sup>1</sup>; <sup>p</sup> 6¼ <sup>q</sup> <sup>j</sup>∀ð Þ <sup>a</sup>; <sup>b</sup> <sup>∈</sup> <sup>A</sup><sup>2</sup>

#### 6. The proposed architecture

Figure 3 shows an overview of the SoC architecture for cognitive agents proposed in this work. As mentioned before, the cognitive model of the CAA (Figure 1) was used as a base model for

Figure 3. Block diagram of the proposed SoC.

the development of this architecture, and hence it can be seen in the image that each CAA level has a corresponding computational module in the SoC.

The modules were designed according to the task executed by the corresponding CAA level. Still referring to Figure 3:


## 7. The RISC Rete Machine

In this section, the proposed processor system level architecture is described. As it was stated in the introduction, this processor is part of the design of a SoC for cognitive agents. The idea is to have this processor working with another unit for planning in the cognitive level and with a plan execution unit in the instinctive level. The knowledge bases should first be compiled into the application specific ISA of the Rete processor and then downloaded into its instruction memory.

The system level architecture here presented is a RISC application specific processor (ASP) whose special purpose ISA was inspired on the description of the Rete algorithm given in [9], the first paper written about the algorithm. The author uses an assembly-like set of instructions to describe the operation of the algorithm. But they serve only as guidelines for a high-level implementation described afterwards in that paper; no hardware implementations are presented.

• TEQA <field>, <constant>: Implements a CTN where <field> is the field to be tested (object, attribute or value) of the fact register and <constant> is the value this field must be equal to.

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

http://dx.doi.org/10.5772/intechopen.79265

209

• FILTER <field1>, <field2>, <comparison>: Compares two fields inside a fact using a given

• MERGE <parent-bm>, <bm>, <am>, <next-join>: Saves in registers the addresses of its parent memories and of the next MERGE instruction. Also, it updates an alpha memory in a right

• TEST <field1>, <premise>, <field2>, <comparison>: Deals with left and right activations of the JN. It triggers an interruption, jumping to a pre-programmed routine that runs through the memories testing whether or not the <field1> compares to <field2> of <premise>-th

• JOIN <lbl>: jumps to a JN (MERGE instruction) defined previously in the code, on a right

• TERM <rule>, <nsubs>: Represents production nodes. It saves the address of the consequence of the matched rule in a register, for further use. <nsubs> is the number of substitutions this rule has, so that it can jump to the last one (for popping the test stack) in the

• SUBST <p1>, <f1>, <p2>, <f2>, <lst>: Uses the matched token to create substitutions for the variable in the consequence. The pairs (<p1>, <f1>) and (<p2>, <f2>) are "coordinates" of occurrences of the same variable in the premises and the consequence, respectively.

comparison operation. If the comparison fails, so does the match.

premise. The <comparison> operation is given in the field comparison.

case where the current fact is to be excluded instead of added.

activation.

Figure 4. Architecture of the Rete processor.

activation.

Inspired by the aforementioned (pseudo-)instructions presented in [9], this work purposes an actual machine for the execution of the Rete matching algorithm whose ISA implements a modified version of the pseudo-instructions presented in Forgy's seminal paper.

The overall processor architecture is shown in Figure 4. The alpha and beta memories are preallocated at compiling time.

The rules are compiled in a sequence of these instructions instead of being used for the creation of the Rete tree in memory. The alpha and beta memories will still exist but the nodes (constant test and join nodes) will be implemented by instructions (Figure 4).

The new fact is stored in a register together with a bit indicating whether it is being added or deleted. The instructions arguments and operation are detailed below:

• FORK <label>: Represents a branch in the tree, with one of the nodes represented by a node instruction immediately after it and the other at the instruction address given by <label>. This address is stacked and the next instruction is fetched. The instruction address corresponding to <label> is popped from stack when a mismatch occurs and the program jumps to it. If the stack is empty, the match failed.

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture http://dx.doi.org/10.5772/intechopen.79265 209

Figure 4. Architecture of the Rete processor.

the development of this architecture, and hence it can be seen in the image that each CAA level

The modules were designed according to the task executed by the corresponding CAA level.

• the cognitive level is implemented by a Graphplan module, responsible for the execution of graph planning algorithms and a module dedicated to the Rete algorithm. The later is

• the instinctive level comprises only a Rete module that is applied to coordinate with the

• the reactive level, which in fact interacts with the environment, must be as general as

In this section, the proposed processor system level architecture is described. As it was stated in the introduction, this processor is part of the design of a SoC for cognitive agents. The idea is to have this processor working with another unit for planning in the cognitive level and with a plan execution unit in the instinctive level. The knowledge bases should first be compiled into the application specific ISA of the Rete processor and then downloaded into its instruction memory. The system level architecture here presented is a RISC application specific processor (ASP) whose special purpose ISA was inspired on the description of the Rete algorithm given in [9], the first paper written about the algorithm. The author uses an assembly-like set of instructions to describe the operation of the algorithm. But they serve only as guidelines for a high-level implementation

Inspired by the aforementioned (pseudo-)instructions presented in [9], this work purposes an actual machine for the execution of the Rete matching algorithm whose ISA implements a

The overall processor architecture is shown in Figure 4. The alpha and beta memories are pre-

The rules are compiled in a sequence of these instructions instead of being used for the creation of the Rete tree in memory. The alpha and beta memories will still exist but the nodes (constant

The new fact is stored in a register together with a bit indicating whether it is being added or

• FORK <label>: Represents a branch in the tree, with one of the nodes represented by a node instruction immediately after it and the other at the instruction address given by <label>. This address is stacked and the next instruction is fetched. The instruction address corresponding to <label> is popped from stack when a mismatch occurs and the program jumps to it. If the

described afterwards in that paper; no hardware implementations are presented.

modified version of the pseudo-instructions presented in Forgy's seminal paper.

test and join nodes) will be implemented by instructions (Figure 4).

deleted. The instructions arguments and operation are detailed below:

has a corresponding computational module in the SoC.

reactive level the execution of a plan; and

used by the former for the SBC-based state space expansion;

possible, and is implemented by a general purpose CPU.

Still referring to Figure 3:

208 Applications of Mobile Robots

7. The RISC Rete Machine

allocated at compiling time.

stack is empty, the match failed.


<lst> indicates whether it is the last substitution for that match or not. If it is, the test stack must be popped to proceed with interrupted tests.

#### 8. The cognitive module

Figure 3 shows that the computational module associated with the cognitive level of the CAA (and from now on referred to as cognitive module) contains a Graphplan module that communicates with a Rete module, whose architecture was presented in the previous section. The objective of the cognitive module is to solve planning problems using graph planning algorithms. The Graphplan algorithm was used as an inspiration for the conception of the architecture of this module.

This module needs to be generic enough to execute not only the Graphplan algorithm, but also algorithms based on planning graphs or that uses Graphplan as heuristic function. Also, it should have two execution modes: the search mode, where a solution for the planning problem is searched, and the monitor mode, the monitors the execution of found plans.

The system level model of this module consists of two general purpose processors with individual instruction memory (i.e. executing independent programs) and a Rete module communicating with each other through a bus. Figure 5 illustrates this architecture.

9. Case study

changed the program counter.

how to modify the facts base in case of a match.

Figure 6. Sequence diagram of the cognitive module operation.

9.2. Simulated tests: Rete module

9.1. Domain definition

The architecture were simulated using a program written in the Scala programming language, using array structures for the instruction, alpha and beta memories, lists for fork and JN test stacks and variables for registers (program counter, direction flags, stacks counters, alpha and beta indices etc.). The dynamics of the program was dictated by the way the instructions

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

http://dx.doi.org/10.5772/intechopen.79265

211

The architecture will be validated using the block world domain example, from [1]. Figure 7 shows the knowledge base that is going to be used in the simulation. The same filter shown in the Move rule could be present in the MoveToTable rule, but it was omitted for simplicity. In spite of the fact that the consequences will not be used here (only the matching procedure is important), one should notice the add and the rem symbols. Those are simply instructions on

The knowledge base shown in Figure 7 is then compiled into the program presented in Figure 8. This program is the input for the simulator, in the form of an array of instruction objects (objects from an instruction class defined inside the simulator code). It then waits for a new fact to come

The tests performed consisted of feeding some facts that are known to cause a match, one by one, to the simulator and check whether the system detects that match. The facts ð Þ logic Cð Þ ; on; A , ð Þ logic Cð Þ ; type; block and ð Þ logic Cð Þ ;state; clear must match the premise of the rule MoveToTable.

in and then executes the program once for that fact, changing the memories accordingly.

The taxonomy of the task this module should execute justifies this architecture. During planning, two main processes should be executed: the expansion of the states and actions graph, and the recursive search for a solution whenever the goal state is found in the graph. With the architecture described it is possible to execute these processes in parallel.

With this approach, when the goal state is found, the solution search can be started while the graph expansion goes on. Hence, even if the found goal set do not produce a solution, the expansion made progress in parallel, until another valid goal set is found, when the search is triggered again. The dynamics of Graphplan was modified for this module: a KBS is responsible for calculating the applicable actions; this KBS is implemented by the Rete module. The sequence diagram in Figure 6 illustrates this operation.

Figure 5. Block diagram of the cognitive module.

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture http://dx.doi.org/10.5772/intechopen.79265 211

Figure 6. Sequence diagram of the cognitive module operation.

#### 9. Case study

<lst> indicates whether it is the last substitution for that match or not. If it is, the test stack

Figure 3 shows that the computational module associated with the cognitive level of the CAA (and from now on referred to as cognitive module) contains a Graphplan module that communicates with a Rete module, whose architecture was presented in the previous section. The objective of the cognitive module is to solve planning problems using graph planning algorithms. The Graphplan algorithm was used as an inspiration for the conception of the architec-

This module needs to be generic enough to execute not only the Graphplan algorithm, but also algorithms based on planning graphs or that uses Graphplan as heuristic function. Also, it should have two execution modes: the search mode, where a solution for the planning prob-

The system level model of this module consists of two general purpose processors with individual instruction memory (i.e. executing independent programs) and a Rete module

The taxonomy of the task this module should execute justifies this architecture. During planning, two main processes should be executed: the expansion of the states and actions graph, and the recursive search for a solution whenever the goal state is found in the graph. With the

With this approach, when the goal state is found, the solution search can be started while the graph expansion goes on. Hence, even if the found goal set do not produce a solution, the expansion made progress in parallel, until another valid goal set is found, when the search is triggered again. The dynamics of Graphplan was modified for this module: a KBS is responsible for calculating the applicable actions; this KBS is implemented by the Rete module. The

lem is searched, and the monitor mode, the monitors the execution of found plans.

communicating with each other through a bus. Figure 5 illustrates this architecture.

architecture described it is possible to execute these processes in parallel.

sequence diagram in Figure 6 illustrates this operation.

Figure 5. Block diagram of the cognitive module.

must be popped to proceed with interrupted tests.

8. The cognitive module

ture of this module.

210 Applications of Mobile Robots

The architecture were simulated using a program written in the Scala programming language, using array structures for the instruction, alpha and beta memories, lists for fork and JN test stacks and variables for registers (program counter, direction flags, stacks counters, alpha and beta indices etc.). The dynamics of the program was dictated by the way the instructions changed the program counter.

#### 9.1. Domain definition

The architecture will be validated using the block world domain example, from [1]. Figure 7 shows the knowledge base that is going to be used in the simulation. The same filter shown in the Move rule could be present in the MoveToTable rule, but it was omitted for simplicity. In spite of the fact that the consequences will not be used here (only the matching procedure is important), one should notice the add and the rem symbols. Those are simply instructions on how to modify the facts base in case of a match.

#### 9.2. Simulated tests: Rete module

The knowledge base shown in Figure 7 is then compiled into the program presented in Figure 8. This program is the input for the simulator, in the form of an array of instruction objects (objects from an instruction class defined inside the simulator code). It then waits for a new fact to come in and then executes the program once for that fact, changing the memories accordingly.

The tests performed consisted of feeding some facts that are known to cause a match, one by one, to the simulator and check whether the system detects that match. The facts ð Þ logic Cð Þ ; on; A , ð Þ logic Cð Þ ; type; block and ð Þ logic Cð Þ ;state; clear must match the premise of the rule MoveToTable.

Figure 7. Knowledge base for the block world domain example.

Figure 9 shows the output of the simulator after feeding it with the fact ð Þ logic Cð Þ ; on; A . This output contains all the instruction executed by the processor for the given fact. As the entry point of the network is the root node and it has three CTNs as children, a FORK is always processed first. For the current fact, the forked address is only taken after the fact is stored in AM1 (and consequently in BM1 too, since the parent BM is a dummy one), when the JN test fails (pc ¼ 5) due to the absence of facts in AM2. It is noteworthy that the instruction at forked address is another fork, because the root node has three children. In the last CTN (pc ¼ 33) the fork stack is empty, and as the test failed, the program finishes.

For ð Þ logic Cð Þ ; type; block the processing mechanism is the same, but the execution path is different, since the fact is going to a different alpha memory.

Figure 8. Code for the Rete network of the block world example.

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

http://dx.doi.org/10.5772/intechopen.79265

213

Figure 9. Output of the simulator for ð Þ logic Cð Þ ; on; A .

When ð Þ logic Cð Þ ;state; clear is added, a match occurs (pc ¼ 22), as can be seen in Figure 10. In this output it is possible to see the JN tests being stacked once a partial match is found (pc ¼ 5). After that (pc ¼ 8), a test fails, but the previously stacked test is not popped yet: there is a FORK at pc ¼ 6, and forks have priority. Also, in this execution four substitutions take place (pc ¼ 23 to 26).

Finally, Figure 11 shows the output of the simulator for the exclusion of the fact ð Þ logic Cð Þ ; on; A . The procedure for exclusion starts as the same as the one for addition: the instructions guide the traverse of the tree looking for a match. The difference is that no changes are made to the memories. Instead, for every activation or match caused by the input fact, the index of the corresponding memory and its position inside that memory are stacked. At the end of the execution, when there are no more forks or tests stacked, the exclusion stack is pop and the elements of the memories given by the indices stored in it are deleted.

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture http://dx.doi.org/10.5772/intechopen.79265 213

Figure 8. Code for the Rete network of the block world example.

Figure 9 shows the output of the simulator after feeding it with the fact ð Þ logic Cð Þ ; on; A . This output contains all the instruction executed by the processor for the given fact. As the entry point of the network is the root node and it has three CTNs as children, a FORK is always processed first. For the current fact, the forked address is only taken after the fact is stored in AM1 (and consequently in BM1 too, since the parent BM is a dummy one), when the JN test fails (pc ¼ 5) due to the absence of facts in AM2. It is noteworthy that the instruction at forked address is another fork, because the root node has three children. In the last CTN (pc ¼ 33) the

For ð Þ logic Cð Þ ; type; block the processing mechanism is the same, but the execution path is

When ð Þ logic Cð Þ ;state; clear is added, a match occurs (pc ¼ 22), as can be seen in Figure 10. In this output it is possible to see the JN tests being stacked once a partial match is found (pc ¼ 5). After that (pc ¼ 8), a test fails, but the previously stacked test is not popped yet: there is a FORK at pc ¼ 6, and forks have priority. Also, in this execution four substitutions take place

Finally, Figure 11 shows the output of the simulator for the exclusion of the fact ð Þ logic Cð Þ ; on; A . The procedure for exclusion starts as the same as the one for addition: the instructions guide the traverse of the tree looking for a match. The difference is that no changes are made to the memories. Instead, for every activation or match caused by the input fact, the index of the corresponding memory and its position inside that memory are stacked. At the end of the execution, when there are no more forks or tests stacked, the exclusion stack is pop and

fork stack is empty, and as the test failed, the program finishes.

Figure 7. Knowledge base for the block world domain example.

different, since the fact is going to a different alpha memory.

the elements of the memories given by the indices stored in it are deleted.

(pc ¼ 23 to 26).

212 Applications of Mobile Robots

Figure 9. Output of the simulator for ð Þ logic Cð Þ ; on; A .

As mentioned early in this chapter, one of the tasks of the cognitive module is to monitor plan execution. But this simulated experiment deals only with the execution of the Graphplan algorithm for planning problem solution, since this is the computationally heavier task that

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

http://dx.doi.org/10.5772/intechopen.79265

215

The problem domain chosen for this test was, as in the Rete module, the block world domain. But this time, for the establishment of a planning problem, a start state and a goal state must be

To show the execution of the experiment, the sequence diagram in the Figure 13 was generated

represents the processor implementing the graph expansion) communicates with the Rete module to generate the state and action layers of the planning graph. This sub-module is also responsible for the calculation of mutex relations in each layer and for detecting the presence of the goal set in the most recent state layer. It can be seen that after three expansion cycles the goal is found and the search sub-module is triggered, starting the recursive search for a

The solution is not found, and this is the point where the main advantage of the architecture became apparent: while the search sub-module tries to find a plan in the graph, the expansion and the Rete modules work together to generate a new layer in the graph. And, for this experiment, the new layer also has the objective state. The search module finds a solution this time, namely: hfmoveToTable Að Þ ; D , moveToTable Cð Þg ; B , move C f g ð Þ ; table; D , move B f g ð Þ ; table; C ,

The simulation generated a sequence of commands corresponding to the task being executed by each sub-module and the messages being exchanged between them, and the commands were transformed in the sequence diagram by the tool

. In the image it is possible to see that the expansion sub-module (that

inspired the architecture.

by the simulation<sup>1</sup>

solution.

1

provided. Those are shown in Figure 12.

Figure 12. Start and goal states for the planning simulation.

f g move Að Þ ; table; B i. The planning is then halted.

WebSequenceDiagrams (https://www.websequencediagrams.com/).

Figure 10. Output of the simulator for ð Þ logic Cð Þ ;state; clear .

Figure 11. Output of the simulator for deleting ð Þ logic Cð Þ ; on; A .

#### 9.3. Simulated tests: cognitive module

For the cognitive module simulation the Akka library was used. The reason is that Akka implements the actor model of parallelism, which allows the program to contain routines running in parallel but with isolated execution contexts, communicating only through message passing; this suits the module specification given in Section 8.

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture http://dx.doi.org/10.5772/intechopen.79265 215

Figure 12. Start and goal states for the planning simulation.

9.3. Simulated tests: cognitive module

Figure 11. Output of the simulator for deleting ð Þ logic Cð Þ ; on; A .

Figure 10. Output of the simulator for ð Þ logic Cð Þ ;state; clear .

214 Applications of Mobile Robots

passing; this suits the module specification given in Section 8.

For the cognitive module simulation the Akka library was used. The reason is that Akka implements the actor model of parallelism, which allows the program to contain routines running in parallel but with isolated execution contexts, communicating only through message As mentioned early in this chapter, one of the tasks of the cognitive module is to monitor plan execution. But this simulated experiment deals only with the execution of the Graphplan algorithm for planning problem solution, since this is the computationally heavier task that inspired the architecture.

The problem domain chosen for this test was, as in the Rete module, the block world domain. But this time, for the establishment of a planning problem, a start state and a goal state must be provided. Those are shown in Figure 12.

To show the execution of the experiment, the sequence diagram in the Figure 13 was generated by the simulation<sup>1</sup> . In the image it is possible to see that the expansion sub-module (that represents the processor implementing the graph expansion) communicates with the Rete module to generate the state and action layers of the planning graph. This sub-module is also responsible for the calculation of mutex relations in each layer and for detecting the presence of the goal set in the most recent state layer. It can be seen that after three expansion cycles the goal is found and the search sub-module is triggered, starting the recursive search for a solution.

The solution is not found, and this is the point where the main advantage of the architecture became apparent: while the search sub-module tries to find a plan in the graph, the expansion and the Rete modules work together to generate a new layer in the graph. And, for this experiment, the new layer also has the objective state. The search module finds a solution this time, namely: hfmoveToTable Að Þ ; D , moveToTable Cð Þg ; B , move C f g ð Þ ; table; D , move B f g ð Þ ; table; C , f g move Að Þ ; table; B i. The planning is then halted.

<sup>1</sup> The simulation generated a sequence of commands corresponding to the task being executed by each sub-module and the messages being exchanged between them, and the commands were transformed in the sequence diagram by the tool WebSequenceDiagrams (https://www.websequencediagrams.com/).

Simulated experiments shown that the architecture proposed for the aforementioned modules are valid in the sense of performing correctly their tasks. As future works, a formal verification method should be applied in order to validate the all modules separately and working together, as well as their computational and energetic performance, to show that the proposed architecture allows for low-level symbolic processing, which can give embedded and on-chip

IntelliSoC: A System Level Design and Conception of a System-on-a-Chip (SoC) to Cognitive Agents Architecture

http://dx.doi.org/10.5772/intechopen.79265

217

systems fast automatic reasoning capabilities, with low energy consumption.

Diego Ferreira\*, Augusto Loureiro da Costa and Wagner Luiz Alves De Oliveira

[1] Russel S, Norvig P. Inteligencia Artificial. Rio de Janeiro: Elsevier; 2004

actions on Evolutionary Computation. 2007;11(2):151

Cognitive Systems Research. 2009;10(2):141-160

Brasileiro de Automao Inteligente; 2013

lem. Artificial Intelligence. 1982;19(1):17-37

CMU-CS-85-126

[2] Huhns MN, Singh MP. Cognitive agents. IEEE Internet Computing. 1998;2(6):87-89

[3] Vernon D, Metta G, Sandini G. A survey of artificial cognitive systems: Implications for the autonomous development of mental capabilities in computational agents. IEEE Trans-

[4] Langley P, Laird JE, Rogers S. Cognitive architectures: Research issues and challenges.

[5] Costa AL, Bittencourt G. From a concurrent architecture to a concurrent autonomous

[6] Bittencourt G, Costa ALd. Hybrid cognitive model. In: Third International Conference on Cognitive Science ICCS'2001 Workshop on Cognitive Agents and Agent Interaction; 2001

[7] Cerqueira RG, Costa ALd, McGill SG, Lee D, Pappas G. From reactive to cognitive agents: Extending reinforcement learning to generate symbolic knowledge bases. In: Simpsio

[8] Ferreira DSF, Silva RRd, Costa ALd. Embedding a concurrent autonomous agent in a microcontrollers network. In: 5th ISSNIP-IEEE Biosignals and Biorobotics Conference (2014): Biosignals and Robotics for Better and Safer Living (BRC). IEEE; 2014. pp. 1-6 [9] Forgy CL. Rete: A fast algorithm for the many pattern/many object pattern match prob-

[10] Lehr TF. The implementation of a production system machine; 1985. Technical Report

agents architecture. Lecture Notes in Artificial Intelligence. 1999;1856:85-90

\*Address all correspondence to: diego.stefano@gmail.com

Federal University of Bahia, Salvador, BA, Brazil

Author details

References

Figure 13. Sequence diagram for the simulation of the cognitive module.

#### 10. Final considerations

This chapter presented the system level design of a SoC for the execution of cognitive agents. It uses the cognitive model of a agent architecture named concurrent autonomous agent (CAA). The SoC architecture was described in terms of the levels of the CAA and the tasks these levels execute. Two main computational modules of the SoC are the RISC Rete machine that employs an application specific ISA to detect matches in a KBS (in both the SoC correspondents of the instinctive and cognitive levels of the CAA) and the multiprocessed cognitive level dedicated to (graph) planning.

Simulated experiments shown that the architecture proposed for the aforementioned modules are valid in the sense of performing correctly their tasks. As future works, a formal verification method should be applied in order to validate the all modules separately and working together, as well as their computational and energetic performance, to show that the proposed architecture allows for low-level symbolic processing, which can give embedded and on-chip systems fast automatic reasoning capabilities, with low energy consumption.

#### Author details

Diego Ferreira\*, Augusto Loureiro da Costa and Wagner Luiz Alves De Oliveira

\*Address all correspondence to: diego.stefano@gmail.com

Federal University of Bahia, Salvador, BA, Brazil

#### References

10. Final considerations

216 Applications of Mobile Robots

Figure 13. Sequence diagram for the simulation of the cognitive module.

to (graph) planning.

This chapter presented the system level design of a SoC for the execution of cognitive agents. It uses the cognitive model of a agent architecture named concurrent autonomous agent (CAA). The SoC architecture was described in terms of the levels of the CAA and the tasks these levels execute. Two main computational modules of the SoC are the RISC Rete machine that employs an application specific ISA to detect matches in a KBS (in both the SoC correspondents of the instinctive and cognitive levels of the CAA) and the multiprocessed cognitive level dedicated


[11] Peters M, Brink C, Sachweh S, Zündorf A. Scaling parallel rule-based reasoning. In: The

[12] Panteleyev MG, Puzankov DV, Kolosov GG, Govorukhin IB. Design and implementation of hardware for real-time intelligent agents. In: 2002 IEEE International Conference on

[13] Frost J, Numan M, Liebelt M, Phillips B. A new computer for cognitive computing. In: 2015 IEEE 14th International Conference on Cognitive Informatics & Cognitive Comput-

[14] Kim Y, Shin D, Lee J, Lee Y, Yoo HJ. A 0.55 V 1.1m Wartificial-intelligence processor with PVT compensation for micro robots. In: 2016 IEEE International Solid-State Circuits Con-

[15] Bittencourt G. The quest of the missing link. In: International Joint Conference of Artificial

[16] Ferreira DSF, Paim CC, Costa ALd. Using a real-time operational system to embed a kinematic controller in an omnidirectional mobile robot. In: XI Simpósio Brasileiro de Automação Inteligente (SBAI) e XI Conferência Brasileira de Dinâmica, Controle e Aplicações (DINCON);

[17] Forgy CL. On the Efficient Implementation of Production Systems. Dissertation. Depart-

Semantic Web: Trends and Challenges. Cham: Springer; 2014. pp. 270-285

Artificial Intelligence Systems (ICAIS 2002). IEEE; 2002. pp. 6-11

ment of Computer Science, Carnegie-Mellon University; 1979

ing (ICCI\* CC). IEEE; 2015. pp. 33-38

ference (ISSCC). IEEE; 2016. pp. 258-259

Intelligence; 1997

218 Applications of Mobile Robots

2013. pp. 1-6

## *Edited by Efren Gorrostieta Hurtado*

This book includes a selection of research work in the mobile robotics area, where several interesting topics are presented. In this way we find a review of multi-agents, different techniques applied to the navigation systems, artificial intelligence algorithms, which include deep learning applications, systems where a Kalman filter estimator is extended for visual odometry, and finally the design of an on-chip system for the execution of cognitive agents. Additionally, the development of different ideas in mobile robot applications are included and hopefully will be useful and enriching for readers.

Published in London, UK © 2019 IntechOpen © PhonlamaiPhoto / iStock

Applications of Mobile Robots

IntechOpen Book Series

Mobile Robotics, Volume 1

Applications of Mobile Robots

*Edited by Efren Gorrostieta Hurtado*