**3.1 Drone outdoor localization using RTK GNSS**

The RTK enhancement feature of GPS is used for outdoor localization purposes. This is due to the more precise positioning [17] because the of the GPS satellite measurements' correction using feedback from an additional stationary GPS module. The disadvantage of such systems is that their use is bounded to a significant pre-flight setup time which is inversely proportional to the achieved accuracy (cm range).

Although the internal loop of the flight controller operates at 400 Hz, the GPS receiver streams data at a lower rate of 5 Hz. In popular flight software such as ArduPilot, the aforementioned rate needs to be taken into consideration by the underlying EKFs running by the FCU. A typical comparison of the achieved accuracy using a drone in a hovering state can be seen in **Figure 5**.

The drone was flown in a hovering position with the RTK GPS module injecting measurements to the flight controller and the output of the FCU's EKF was compared with and without the presence of the injected RTK measurements. The red line represents the EKF's output based solely on the GPS signal, whilst the blue line indicates the same output when RTK correction (using a 30 min warmup period) is injected on the FCU.

The standard deviation was computed equal to 0.74 m, 0.47 m and 0.27 m for *X* , *Y* and *Z* respectively when no RTK correction was applied. Contrary to this, the same values with RTK injection were computed to equal 0.05 m, 0.02 m and 0.23 m respectively. It should be noted that there is no significant improvement in the *Z* -direction, indicating the need to use either a barometer or a laser sensor for ground clearance measurements.

### **3.2 Drone indoor localization**

During indoor navigation: a) the lack of GPS guidance, b) pressure changes affecting the barometric sensor, and c) power lines affecting compass accuracy can severely affect the output of a FCU. With only the accelerometers and gyroscopes being unaffected, the injection of an external feedback source to the FCU is considered essential. Such feedback is usually based on visual techniques, such as those presented in [18, 19].

**Figure 5.** *Drone's EKF 3D-position output with (red) and without (blue) RTK correction.*

For experimentation purposes, the used Motion Capture System (MoCaS) [20] injects measurements in the ArduCopter flight stack. The system comprises of 24 Vicon cameras uniformly scattered within an orthogonal space of 15 5 8 ´´= ´ ´ (*LW H*) m. The utilized system allows simultaneous tracking of 100 objects at 120 Hz with sub-millimeter accuracy. Despite MoCaS's high refresh rate, the ArduCopter flight stack at the FCU accepts external positioning data at a 4 Hz streaming rate.

The utilized ROS software at the MoCaS operates at 25 Hz and can efficiently wirelessly stream the measurements to the drone's FCU. The latency time *C FCU d V C FCU ttt t* =+ + , where ( ) *C FCU V C t t* is the delay of data streaming to the companion computer (FCU), and *FCU t* the delay of processing the data on the FCU. In the developed system typical measured values are 5 *<sup>C</sup> Vt ms* , 20 *FCU Ct ms* = and *t ms FCU* = 40 , resulting in *t ms <sup>d</sup>* 75 .

Because of the MoCaS's efficiency, its weighing to the EKF is ten times larger compared to the GPS's weight when flying outdoors. Subsequently, the efficiency of the implementation is assessed by comparing EKF's position output with and without MoCaS's feedback injection. In **Figure 6** the drone's position error (in each axis) between the aforementioned two quantities is visualized, where the red, green and blue lines represent the error along the *X Y* and *Z* axes respectively.

Real time pose tracking is satisfactorily achieved and minor differences are attributed to the EKF's weighting of the accelerometer and gyroscope measurements during calculations.

### **4. Drone awareness of surrounding environment**

An important parameter on aerial navigation is awareness of the surrounding environment including being in close proximity between cooperating or evasive drones [21, 22] to avoid potential contacts.

High accuracy awareness may not be feasible [23] and can become prohibitive in indoor environments; visual sensors along with Lidars can assist in this aspect. A spherical camera provides an all-around visualization of the surroundings and

**45**

**Figure 7.**

*Spherical flat image.*

*Development of a Versatile Modular Platform for Aerial Manipulators*

**4.1 Environment awareness using a spherical camera**

can detect neighboring targets. A Pan-Tilt-Zoom (PTZ) camera with a limited Field of View (FoV) can then provide a more accurate description of this target. The suggested target relies on the detection of moving objects. Correlation techniques and/or deep learning Visual Object Tracking (VOT) methods [24] are employed for

Rather than using several cameras with a limited Field of View (FoV) to observe

For the case of collaborating drones, it is assumed that each one carries passive markers for visual recognition. Subsequently, the rectified images are processed for identification of these markers [28–31] thus estimating the neighboring drone's pose. For improved pose extraction, the solution of a multi-marker rhombicuboctahedron formation arrangement [32] is assumed to be present in each target.

The experimental setup for evaluation consists of the spherical camera mounted in a 2.7 m protruding stick, which subsequently is mounted to the underside of the octarotor using the generic mount base discussed in Section 5. A rhombicuboctahedron arrangement with markers at its faces is attached to a DJI-Mavic drone. Both UAVs were located within the MoCaS test volume, as shown in **Figure 9**. The quadrotor drone was flown in a randomized trajectory near the vicinity of the octarotor.

In **Figure 10** the relative 3D-flight path between the drones is presented. The results recorded from the MoCaS and the visual ones are shown, where for the cases of detecting the marker the relative accuracy these measurements was 2.2 cm respectively.

the surrounding space, a 360° FoV camera [25] is used. The spherical camera records images in a "spherical format" which is comprised of two wide-angle frames stitched together to form a virtual sphere [25]. The image can be rectified to the classic distortionless rectilinear format of a pinhole camera [26]. However, due to the nature of the "spherical format," it is preferable to split the image into smaller segments and rectify each one to achieve results closer to the pinhole camera model. Instead of splitting into equal sized square segments [27], each image is split into tiles based on orientation-independent circles. With every tile having a different a-priori known calibration, the rectification can be carried out for each one independently, without high computational cost. By applying the solution and rectifying the image in **Figure 7**, for a selection of *N* = 12 tiles, the resulting rectified

*DOI: http://dx.doi.org/10.5772/intechopen.94027*

partitions are visualized in **Figure 8**.

this purpose.

**Figure 6.** *Drone's EKF position error when MoCaS' measurements are not injected.*

*Service Robotics*

streaming rate.

*C FCU*

*d V C FCU ttt t* =+ + , where ( ) *C FCU*

*t ms FCU* = 40 , resulting in *t ms <sup>d</sup>* 75 .

ments during calculations.

developed system typical measured values are 5 *<sup>C</sup>*

**4. Drone awareness of surrounding environment**

*Drone's EKF position error when MoCaS' measurements are not injected.*

drones [21, 22] to avoid potential contacts.

For experimentation purposes, the used Motion Capture System (MoCaS) [20] injects measurements in the ArduCopter flight stack. The system comprises

15 5 8 ´´= ´ ´ (*LW H*) m. The utilized system allows simultaneous tracking of 100 objects at 120 Hz with sub-millimeter accuracy. Despite MoCaS's high refresh rate, the ArduCopter flight stack at the FCU accepts external positioning data at a 4 Hz

The utilized ROS software at the MoCaS operates at 25 Hz and can efficiently

Because of the MoCaS's efficiency, its weighing to the EKF is ten times larger compared to the GPS's weight when flying outdoors. Subsequently, the efficiency of the implementation is assessed by comparing EKF's position output with and without MoCaS's feedback injection. In **Figure 6** the drone's position error (in each axis) between the aforementioned two quantities is visualized, where the red, green and blue lines represent the error along the *X Y* and *Z* axes respectively. Real time pose tracking is satisfactorily achieved and minor differences are attributed to the EKF's weighting of the accelerometer and gyroscope measure-

An important parameter on aerial navigation is awareness of the surrounding environment including being in close proximity between cooperating or evasive

High accuracy awareness may not be feasible [23] and can become prohibitive in indoor environments; visual sensors along with Lidars can assist in this aspect. A spherical camera provides an all-around visualization of the surroundings and

*V C t t* is the delay of data streaming to the companion

*Vt ms* , 20 *FCU*

*Ct ms* = and

of 24 Vicon cameras uniformly scattered within an orthogonal space of

wirelessly stream the measurements to the drone's FCU. The latency time

computer (FCU), and *FCU t* the delay of processing the data on the FCU. In the

**44**

**Figure 6.**

can detect neighboring targets. A Pan-Tilt-Zoom (PTZ) camera with a limited Field of View (FoV) can then provide a more accurate description of this target. The suggested target relies on the detection of moving objects. Correlation techniques and/or deep learning Visual Object Tracking (VOT) methods [24] are employed for this purpose.
