**2. Materials and methods**

Before starting with the implementation of the artificial vision system, it was necessary to carry out a general adjustment of the robot. This process has several stages which involve hardware conditioning and considerable work in software development. The methodology addressed in this chapter is presented in **Figure 1**.

In this section, the kind of robot arm, camera, and the welding system are described in a deeper way. In addition, the methodology of the development of each stage of the project is explained.

*Implementation of an Artificial Vision System for Welding in the Retrofitting Process… DOI: http://dx.doi.org/10.5772/intechopen.88360*

**Figure 1.** *Methodology description.*

### **2.1 Robot arm**

arms; this is normally due to the robot programmer who plans the given robot task, which is executed cyclically [7]. However, some authors present approaches where they implemented different tasks using artificial vision; an example is presented in [8], where the artificial vision is shown as an alternative to guarantee the security of the manipulator arms against collisions, instead of using sensors. On the other hand, in [9], it is shown how an object manipulator robot, which is able to distinguish between different pieces of cutlery, was designed; they divided their work in two stages: the first stage includes the recognition of the objects through computer vision algorithms and finding the coordinates of the objects; and the second stage is the realization of the movement of the robot to the found coordinates using the forward kinematic, and in addition, by means of descent gradient method. A similar approach was shown in [10], where the authors presented a prototype of a system that classifies the rocks that feed the grinding process in a concentration plant; in this case, the researchers use a 2D artificial vision system that estimates the size of the rocks based on their area and distinguishes them; nevertheless, their results are ambiguous because they did the images acquisition statically in a dynamic system. In the case of the welding robot arms, when we can integrate technologies like artificial vision and computational intelligence with industrial robot, it is possible to adapt us to multiple environments and different types of processes, which are very attractive. The principal reason to do this implementation is to search a reduction in production times and an increase in the efficiency of the welding process. Developments as discussed earlier in this chapter take strength because most of the technologies currently available are very expensive for medium and small companies. The aim of this chapter is to include an artificial vision system in the retrofitting

robotic arm process. For this purpose, an old robot manipulator supplied by a company was used, and its behavior was checked. In the general review of the system, it was observed that it cannot make complex moves due to failures in the system, associated mainly to encoders of the servomotors. A new panel control was designed, and in addition, an open source software was implemented to integrate the welding process with artificial vision, in order to locate spatially the working pieces. In this section, the kind of robot arm, camera, and the welding system are described in a deeper way. The methodology of each stage of the project is explained, based on the robot arm kinematic and the generation of a G code with the intersection points of the pieces. A similar work as proposed in this chapter is presented in [1], where the retrofitting of an industrial robot is developed; this work is specially related with ours because the authors showed a modernization of an old robot arm, and besides, they did a comparative study related with the numerical control machines controllers integration using LinuxCNC controller and MatLab/Mach3, where they concluded that LinuxCNC is a better solution because it needs few hardware resources in comparison with Mach3/MatLab alternative; in addition, the license does not have any cost, however this work does not use

Before starting with the implementation of the artificial vision system, it was necessary to carry out a general adjustment of the robot. This process has several stages which involve hardware conditioning and considerable work in software development. The methodology addressed in this chapter is presented in **Figure 1**. In this section, the kind of robot arm, camera, and the welding system are described in a deeper way. In addition, the methodology of the development of each

artificial vision in the positioning of the tool.

**2. Materials and methods**

*Digital Imaging*

stage of the project is explained.

**64**

An MR-2000 welding robot Miller manufactured in 1995 is used for the development project, as shown in **Figure 2**. The robot spent several years deteriorating, and at the time of the evaluation, several errors in the motherboard and the encoders of the servomotors were found. This made impossible to use the original hardware and software.

The arm counts with a morphology of a palletizing robot, 6 freedom grades, and an approximated weight of 353 lb (160 kg). Due to the bad condition of the robot, a total rebuild of the control panel was made; for these modifications, several components were required and the most representative of them are exposed in **Table 1**.

Taking as reference the architecture of palletizing robot presented in [11], the control of the heaviest parts of the robot (base, big forward arm, and big back arm) is designed so that three servomotors of great power (1.0 kW) are responsible for global robot movements. Another three servomotors of 0.12 kW control the remaining pieces which are more focused in the final tool orientation (forearm, wrist, and a grabber). Also, the control includes computational hardware that contains the OS Linux and the vision system for the location and control of the arm.

#### **2.2 Software-robot link**

An intensive search was made for the selection of the most appropriate software for arm control; taking that into count, we realized that some of the programs in the middle are made by people who do not find a good option for this application;

**Figure 2.** *Welding robot Miller.*


have a setup serial where each articulation depends on the previous articulation. For the case of the robots type serial, the system can be defined by Denavit-Hartenberg parameters [13]. This approach uses homogeneous transformations by means of matrices, as this method is frequently used. Our robot is not a serial robot. The MR-2000 has a palletized [11, 14] extension that belongs to the joint 3. By this motive, it was necessary to develop a model kinematics which is able to work with that

*Implementation of an Artificial Vision System for Welding in the Retrofitting Process…*

The movements of a robotic arm are described by two types of kinematics,

• *Forward kinematics*: this is a procedure to get the coordinates in our reference plane [X*T*, Y*T*, Z*T*, A, B] taking as reference the angular positions of each joint

• *Inverse kinematics*: this principally seeks to find the angular positions [*θ*1, *θ*2, *θ*3, *θ*4, *θ*5] of each joint based on spatial coordinates [X*T*, Y*T*, Z*T*, A, B] given. In this case, multiple solutions are possible. Therefore, it is quite important to define the restrictions of the robot. This kind of kinematics is worked in

of the system [*θ*1, *θ*2, *θ*3, *θ*4, *θ*5]. This kind of kinematics is worked in

geometry.

**Figure 3.**

**Figure 4.**

**67**

*Side view of the robot.*

*Top view of the robot.*

forward kinematics and inverse kinematics.

*DOI: http://dx.doi.org/10.5772/intechopen.88360*

LinuxCNC from joint mode.

LinuxCNC from world mode.

#### **Table 1.**

*Materials.*

therefore, they choose to design and build their own software. These approaches have an open architecture but with much work to develop and some are unavailable.

The software used to control the robot arm is designed to CNC type machines, however, it can be implemented the kinematics of different kind of robots and all the instructions and protocols to move until a 9-Axis robot. LinuxCNC is a software widely known in robotic applications and has several advantages in respect to others [1]. It is integrated with an interface to control and monitor each move of the servomotor in a join or world mode. The software was configured using the MESA i525; this device is a general purpose FPGA based programmable I/O card for the PCI bus, and it uses standard parallel port pinouts and connectors for compatibility with most parallel port interfaced motion control/CNC breakout cards/multi-axes step motor drives; for this interface, a new parallel breakout board is designed by means of a digital optocoupled connection between the mesa 5i25 and the servodrivers; the capacitive sensor which are considered as the home switch of the robot also receives signals; this sensor is the physical mean to enable all the arm controls and has the connection of the stop control that disables the robot's hardware and software. Two of these boards are implemented, the first one for the axes 0, 1, and 2 and the other one for the axes 3, 4, and 5 distributed by the type of servodriver that drives each axis [12]. They have the compatibility to receive a numerical program (G code) for the movement of the robot, which makes possible the integration between the control of the robot and the algorithm that uses computer vision to locate the working pieces, and it generates the spacial coordinates XYZ of the welding points through a G code loaded in LinuxCNC. In order to set the MESA 5i25 for working with our electronic boards that were previously made, a custom firmware is installed in the board, where the pin inputs and pin outputs are set based on the pin distribution of our control system.

It is necessary to make the initial configuration of the machine in PNCconf of LinuxCNC, that is a helped interface which leads to user in the set up of machines which use MESA boards; however, PNCconf has several limitations from its interface, and usually, it is necessary to make modifications in the principal configuration files created by this software; those files have the *:ini* and *:hal* extensions, respectively. Normally, the modifications in the configuration files are related to the number of joint of the machine, because the interface is not possible to set more than four of them.

#### **2.3 Kinematics development**

Kinematics is a common problem in the development of software for the manipulation of the robotics arm due to math complexity. Typically, robotic arms *Implementation of an Artificial Vision System for Welding in the Retrofitting Process… DOI: http://dx.doi.org/10.5772/intechopen.88360*

have a setup serial where each articulation depends on the previous articulation. For the case of the robots type serial, the system can be defined by Denavit-Hartenberg parameters [13]. This approach uses homogeneous transformations by means of matrices, as this method is frequently used. Our robot is not a serial robot. The MR-2000 has a palletized [11, 14] extension that belongs to the joint 3. By this motive, it was necessary to develop a model kinematics which is able to work with that geometry.

The movements of a robotic arm are described by two types of kinematics, forward kinematics and inverse kinematics.


**Figure 4.** *Side view of the robot.*

therefore, they choose to design and build their own software. These approaches have an open architecture but with much work to develop and some are unavailable. The software used to control the robot arm is designed to CNC type machines, however, it can be implemented the kinematics of different kind of robots and all the instructions and protocols to move until a 9-Axis robot. LinuxCNC is a software widely known in robotic applications and has several advantages in respect to others [1]. It is integrated with an interface to control and monitor each move of the servomotor in a join or world mode. The software was configured using the MESA i525; this device is a general purpose FPGA based programmable I/O card for the PCI bus, and it uses standard parallel port pinouts and connectors for compatibility with most parallel port interfaced motion control/CNC breakout cards/multi-axes step motor drives; for this interface, a new parallel breakout board is designed by

**Description Quantity** Servomotor 880-DST-A6HKB 3 Servomotor 11A-DST-A6HKB 3 DYN4-H01A2-00 3 DYN2-TLA6S-00 3 Computational Hardware 1 I25 Superport FPGA based PCI 1 Artificial vision system 1

means of a digital optocoupled connection between the mesa 5i25 and the

set based on the pin distribution of our control system.

than four of them.

**66**

**Table 1.** *Materials.*

*Digital Imaging*

**2.3 Kinematics development**

servodrivers; the capacitive sensor which are considered as the home switch of the robot also receives signals; this sensor is the physical mean to enable all the arm controls and has the connection of the stop control that disables the robot's hardware and software. Two of these boards are implemented, the first one for the axes 0, 1, and 2 and the other one for the axes 3, 4, and 5 distributed by the type of servodriver that drives each axis [12]. They have the compatibility to receive a numerical program (G code) for the movement of the robot, which makes possible the integration between the control of the robot and the algorithm that uses computer vision to locate the working pieces, and it generates the spacial coordinates XYZ of the welding points through a G code loaded in LinuxCNC. In order to set the MESA 5i25 for working with our electronic boards that were previously made, a custom firmware is installed in the board, where the pin inputs and pin outputs are

It is necessary to make the initial configuration of the machine in PNCconf of LinuxCNC, that is a helped interface which leads to user in the set up of machines which use MESA boards; however, PNCconf has several limitations from its interface, and usually, it is necessary to make modifications in the principal configuration files created by this software; those files have the *:ini* and *:hal* extensions, respectively. Normally, the modifications in the configuration files are related to the number of joint of the machine, because the interface is not possible to set more

Kinematics is a common problem in the development of software for the manipulation of the robotics arm due to math complexity. Typically, robotic arms

An approximation to the structure of the robot is presented in **Figures 3** and **4**, where it is possible to observe the different parameters involved in the kinematics of the robot.

*B* ¼ *θ*<sup>5</sup> (6)

(7)

2

*XT* and *YT* are located in **Figure 3**, and *ZT*, A, and B in **Figure 4**. Based on the above, the LinuxCNC system can find the final position of the robot in the system of

*Implementation of an Artificial Vision System for Welding in the Retrofitting Process…*

In order to find the values of angular positions based on the spatial coordinates,

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

vuuuut (8)

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi *X*2 *<sup>T</sup>* <sup>þ</sup> *<sup>Y</sup>*<sup>2</sup> *T*

> *L*3 ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi *a*2 <sup>2</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> 3

*sin A* � *arccos <sup>L</sup>*<sup>3</sup> ffiffiffiffiffiffiffiffiffi

1

1

1

2

þ ð Þ *b* � *a*<sup>0</sup>

1

CCA

1

CA

(10)

CA

� � (9)

*a*2 2þ*L*<sup>2</sup> 3

> *a*2 <sup>1</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> 2

q

CA (11)

p � � � �

CA � *arcsin <sup>a</sup>*<sup>1</sup> ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

*θ*<sup>4</sup> ¼ *A* � *θ*<sup>3</sup> (12) *θ*<sup>5</sup> ¼ *B* (13)

0

B@

CA

CA

q

0

B@

*XT YT*

*h*

1

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi *a*2 <sup>1</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> 2 q <sup>2</sup>

1

For the execution of a welding process, it is necessary that the robotic arm meets a series of requirements related to the precision and straightness in the movement of the axes. These parameters are necessary for taking the robot to work under the welding standards of parts. To verify the precision in the displacement of the axes, a dial comparator is placed as shown in **Figure 5** and the moving each axis is mea-

Commands by MDI (Command Line Interface) was sent to test the move of the robotic arm and the kinematics. The result of these moves is shown in the **Table 3**, where the precision of the arm is measured with a digital dial comparator with a resolution of 0.01 mm. The results show a maximum difference of 0.16 mm being permissible for the application in which it will be employed. Each axis is assembled as in **Figure 5**, in which the indicator is located parallel and centered to the axis to

*b* ¼

0

B@

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi *a*2 <sup>1</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> 2 q <sup>2</sup>

q

2*hL*<sup>1</sup>

*<sup>h</sup>*<sup>2</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> <sup>1</sup> �

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi *a*2 <sup>1</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> 2 q <sup>2</sup>

2*h*

sured, so that the following results are obtained.

q

*ZT* � *d*<sup>0</sup> þ

0

B@

q

*sin A* � *arccos*

*θ*<sup>1</sup> ¼ *arctan*

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi *a*2 <sup>2</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> 3

2*hL*<sup>1</sup>

<sup>þ</sup> *<sup>h</sup>*<sup>2</sup> � *<sup>L</sup>*<sup>2</sup> 1

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi *a*2 <sup>1</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> 2 1

CA

coordinates [X*T*, Y*T*, Z*T*, A, B] in the joint mode.

ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi *a*2 <sup>2</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> 3

q

*<sup>h</sup>*<sup>2</sup> <sup>þ</sup> *<sup>L</sup>*<sup>2</sup> <sup>1</sup> �

0

B@

0

BB@

0

B@

þ *arcsin*

*θ*<sup>3</sup> ¼ �*θ*<sup>2</sup> þ *arccos*

� *arccos*

be measured.

**69**

the following equations are developed:

*DOI: http://dx.doi.org/10.5772/intechopen.88360*

*ZT* � *d*<sup>0</sup> þ

*3.1.2 Inverse kinematics*

0

B@

*θ*<sup>2</sup> ¼ *arccos*

*h* ¼

LinuxCNC has a module of kinematics that allows loading a file of code C++ previously compiled with the model of the kinematics of the robot. In this file, both kinematics should be defined for the complete functioning of the robot.
