*3.4.1 SMACH*

Finite state machine (FSM) is a mathematical model of computational science. The objects it represents can be in a limited number of states at any one time. When an external input occurs, the system responds to the external input, and the FSM can conditionally transition from one state to another. This process is called an action. In computer science, finite state machines are widely used for modeling application behavior, hardware circuit system design, software engineering, compilers, network protocols, and computation and language research. FSM can be defined by the present state, condition, action, and substate. The specific interpretation is as follows:


As shown in **Figure 11**, a task can be represented by a state transition diagram. SMACH [13–15], which refers to "State Machine," is a powerful and scalable Python-based library for hierarchical state machines. The SMACH library does not depend on ROS and can be used in any Python project. The *executive\_smach stack,* however, provides very nice integration with ROS, including smooth *actionlib* integration and a powerful SMACH viewer to visualize and introspect state machines. The SMACH core library is lightweight and mainly provides two interfaces: State and Container.

State: The state represents the state being executed. Each state has some potential outputs. The State class outputs the result by implementing the blocking function *execute()*.

Container: The container is a collection of one or more states that enforces some strategy. The simplest container is a State machine. A SMACH state machine can be viewed as a state flow graph, where each node is an execution state (the robot is performing a certain action), and the edges connecting the nodes represent transitions between states. The State machine itself can also be regarded as a state and has its own output, so they can be combined in layers to complete a complex task.

**Figure 12** shows an example state machine [16].

**119**

**Figure 12.**

*SMACH state machine example.*

the message format provided by ROS.

to process the data returned by the *actionlib* server.

*Manipulating Complex Robot Behavior for Autonomous and Continuous Operations*

SMACH uses action files to define communication protocols between different states. The structure of the action file is simple and clear, as shown in **Figure 13**. Three data definition areas are separated by three underscores. The first area defines the message format of the request, and the middle area defines the returned result (result message format, the bottom area defines the intermediate information feedback) message format. Each area can contain multiple data type, and the system will automatically compile the action file into three message files during the compilation process, so the message format for communication between states is actually

SMACH provides a general state type to support invocation while providing a special state class *SimpleActionState* as a proxy for *actionlib.* During the construction of the *SimpleActionState* object, the corresponding *actionlib* client is started by default. The user can define a goal in the constructor and create a callback function

*DOI: http://dx.doi.org/10.5772/intechopen.92254*

**Figure 11.** *Finite state machine.*

*Manipulating Complex Robot Behavior for Autonomous and Continuous Operations DOI: http://dx.doi.org/10.5772/intechopen.92254*

**Figure 12.** *SMACH state machine example.*

SMACH uses action files to define communication protocols between different states. The structure of the action file is simple and clear, as shown in **Figure 13**. Three data definition areas are separated by three underscores. The first area defines the message format of the request, and the middle area defines the returned result (result message format, the bottom area defines the intermediate information feedback) message format. Each area can contain multiple data type, and the system will automatically compile the action file into three message files during the compilation process, so the message format for communication between states is actually the message format provided by ROS.

SMACH provides a general state type to support invocation while providing a special state class *SimpleActionState* as a proxy for *actionlib.* During the construction of the *SimpleActionState* object, the corresponding *actionlib* client is started by default. The user can define a goal in the constructor and create a callback function to process the data returned by the *actionlib* server.

*Service Robotics*

and Container.

complex task.

function *execute()*.

1.Present state: The current state.

state, or it can be terminated.

2.Condition: The premise of triggering an action can also be considered as an

3.Action: The operation performed when the conditions are met and can be regarded as a unit of calculation or transaction processing. After the action is completed, it can be transferred to a new state, it can still be in the original

4.Substate: The state after the present state transition. When different actions occur and different conditions are generated, a state may transition to a different substate. Once the transition is completed, it becomes the present state.

As shown in **Figure 11**, a task can be represented by a state transition diagram. SMACH [13–15], which refers to "State Machine," is a powerful and scalable Python-based library for hierarchical state machines. The SMACH library does not depend on ROS and can be used in any Python project. The *executive\_smach stack,* however, provides very nice integration with ROS, including smooth *actionlib* integration and a powerful SMACH viewer to visualize and introspect state machines. The SMACH core library is lightweight and mainly provides two interfaces: State

State: The state represents the state being executed. Each state has some potential outputs. The State class outputs the result by implementing the blocking

**Figure 12** shows an example state machine [16].

Container: The container is a collection of one or more states that enforces some strategy. The simplest container is a State machine. A SMACH state machine can be viewed as a state flow graph, where each node is an execution state (the robot is performing a certain action), and the edges connecting the nodes represent transitions between states. The State machine itself can also be regarded as a state and has its own output, so they can be combined in layers to complete a

event. When a condition is filled, an action will be triggered.

**118**

**Figure 11.** *Finite state machine.*

**Figure 13.** *File format of action.*

#### *3.4.2 Hierarchical concurrent state machine design*

For the picking robot to perform a continuous picking task, we can use the state transition process to describe it. The detailed description of the state transition is as follows: first enter the startup state, start the picking robot platform, wait for the initialization of each component, and perform a startup self-test. If any component fails to initialize and is in a fault state, the startup fails, and then the system is placed in the error state, stopping working and the task ends.

If the startup is successful, the platform moves into the state, the mobile platform moves to the first picking point, and data collection state and the tomato scanning state are started at the same time. The data collection state collects tomato information in the current status and uploads it to the database of the server; the tomato scan state checks whether there are tomatoes suitable for picking in the viewing area. If not, restart the platform moving state and move the robot to the next picking point; if so, first analyze all the tomato position information and pass the spatial attitude information of the first target tomato to the kinematics solution state according to the predetermined rules, and then program performs kinematics calculation and motion planning. If the tomato is unreachable, the information of the next target tomato is passed to the kinematics solution state, and so on until the last tomato is reached. If it is determined that the tomato is unreachable, the platform is moved to the next picking point. If it is judged that the tomato is reachable, the calculated right arm motion information is transmitted to the robot motion state, and this state sends the trajectory information of the right arm motion to the lower computer and simultaneously detects the joint motion position information during the execution of the lower computer. After the right arm moves to the target position, it enters the suction state. The system starts the suction device to fix the tomatoes and move them to a suitable position, which is convenient for the left arm to cut hands. The kinematics solution state is started again, and the left arm motion information is solved and transmitted to the robot motion state to control the left arm to move near the target. The start of the visual servoing state is close to the target tomato precisely, and the cutting state is started after the arrival, the pneumatic shear transposition of the left arm is started, and the tomato is cut. After completing a tomato pick, pick the next goal planned. Repeat until the last picking point.

According to the task execution process and state transition process described above, the designed state transition diagram is shown in **Figure 14**.

In SMACH, we use *SimpleActionState* to directly simulate the server side of *actionlib* and define a state machine with 10 states to control the robot to complete a comprehensive picking job.

**121**

**Figure 14.**

*Continuous picking status flow.*

*Manipulating Complex Robot Behavior for Autonomous and Continuous Operations*

*DOI: http://dx.doi.org/10.5772/intechopen.92254*

2.DO\_MOVEBASE: mobile platform mobile status.

4.DO\_SPATIAL\_TEMPORAL: data collection status.

5.DO\_KINEMATICS: state of kinematics solution.

7.DO\_MOVE\_ENDEFFECTOR: end effector status.

10.DO\_VISUALSERVOING: visual servoing status.

6.DO\_MOVE\_ROBOT: left and right arm movement status.

Based on the software framework design and operation requirements of the dual-arm picking robot, based on the functional division of each part, a vision module, an eye-hand coordination module, and a task planning module are mainly designed and installed in the industrial computer. Each module further refines

3.DO\_TOMATO\_SCAN: tomato scanning status.

1.DO\_START: start state.

8.DO\_ERROR: fault status.

**3.5 Major software node design**

9.DO\_STOP: emergency stop status.

*Manipulating Complex Robot Behavior for Autonomous and Continuous Operations DOI: http://dx.doi.org/10.5772/intechopen.92254*

1.DO\_START: start state.

*Service Robotics*

**Figure 13.** *File format of action.*

*3.4.2 Hierarchical concurrent state machine design*

placed in the error state, stopping working and the task ends.

above, the designed state transition diagram is shown in **Figure 14**.

In SMACH, we use *SimpleActionState* to directly simulate the server side of *actionlib* and define a state machine with 10 states to control the robot to complete a

For the picking robot to perform a continuous picking task, we can use the state transition process to describe it. The detailed description of the state transition is as follows: first enter the startup state, start the picking robot platform, wait for the initialization of each component, and perform a startup self-test. If any component fails to initialize and is in a fault state, the startup fails, and then the system is

If the startup is successful, the platform moves into the state, the mobile platform moves to the first picking point, and data collection state and the tomato scanning state are started at the same time. The data collection state collects tomato information in the current status and uploads it to the database of the server; the tomato scan state checks whether there are tomatoes suitable for picking in the viewing area. If not, restart the platform moving state and move the robot to the next picking point; if so, first analyze all the tomato position information and pass the spatial attitude information of the first target tomato to the kinematics solution state according to the predetermined rules, and then program performs kinematics calculation and motion planning. If the tomato is unreachable, the information of the next target tomato is passed to the kinematics solution state, and so on until the last tomato is reached. If it is determined that the tomato is unreachable, the platform is moved to the next picking point. If it is judged that the tomato is reachable, the calculated right arm motion information is transmitted to the robot motion state, and this state sends the trajectory information of the right arm motion to the lower computer and simultaneously detects the joint motion position information during the execution of the lower computer. After the right arm moves to the target position, it enters the suction state. The system starts the suction device to fix the tomatoes and move them to a suitable position, which is convenient for the left arm to cut hands. The kinematics solution state is started again, and the left arm motion information is solved and transmitted to the robot motion state to control the left arm to move near the target. The start of the visual servoing state is close to the target tomato precisely, and the cutting state is started after the arrival, the pneumatic shear transposition of the left arm is started, and the tomato is cut. After completing a tomato pick, pick the next goal planned. Repeat until the last picking point. According to the task execution process and state transition process described

**120**

comprehensive picking job.

