*4.1.2 Data collection*

The training images are taken by a small script on the Pi which allows a photo to be taken every X seconds. This was set to 0.8. The image resizing and image filters were added and then implemented; while this was running, the car was moved round the track allowing for multiple angles and directions. All of these photos were then stored on the Pi.

From that set of photos, manual collation was required to store each picture in its corresponding class folder, in this case, one of 0, 1, or 2.

Once these images had been correctly stored, another part of the program could read them from their containing class folder and compute the following to store them in an XML file named Train\_data.xml.

This data is all then stored in Training\_data.xml. The same process occurs again for generating/gathering of training data.

This process occurs to convert the real-time image into a format that is suitable for a neural network's input Layer (**Figure 11**).

**315**

*Computational Efficiency: Can Something as Small as a Raspberry Pi Complete…*

errors in recognising and classifying the image via the NN.

The image is filtered using the image processing techniques (black and white, Gaussian blur, canny edge detection) in order to make the image as simple as possible and have only the lines as a defining feature; this means there is less chance of

The images are scaled down to 10x10 in order to make an efficient and enough neural network; the larger the image, the larger the size of the neural network will need to be, which is directly proportionate to the efficiency/iterations per second of the image classifying loop that the device can perform. Thus, a smaller image resolution was chosen in order to make the device as efficient as possible.

Firstly, the filtered image is retrieved from the live stream from the camera and reshaped row by row to create a 1x100 image and converted to a one-dimensional array of 100 values ranging from 0 to 255 to represent each pixel of the image by each particular element of the array. The one-dimensional array is then directly fed into the neural network as the correct input format. Provided this process occurs on each data item, it can then be fed into the network for training, testing and predic-

Training the network is done via the Train() function within OpenCV. Prior to calling this, the associated parameters have to be set. The process of doing this is relatively simple due to the helpful documentation provided in the OpenCV source codes and thus drastically reduces the amount of code required to show this.

Firstly, all values from training data set need to be loaded and added to the stack

Firstly, the activation function must be set, due to the fact that this network is a

Following this, the network needs to be defined and its topology declared

Multilayer-perceptron, the chosen activation function is sigmoid (**Figure 14**).

*DOI: http://dx.doi.org/10.5772/intechopen.88342*

tion purposes.

(**Figure 13**).

**Figure 12.**

**Figure 13.**

**Figure 14.**

*Loading the data from XML file.*

*Declaring the network and assigning its topology.*

*Set the activation function to sigmoid.*

**4.2 Training the network**

in their respective math formats (**Figure 12**).

Next, the relevant network parameters need to be set.

#### **Figure 11.**

*The process of generating training and test data sets.*

*Computational Efficiency: Can Something as Small as a Raspberry Pi Complete… DOI: http://dx.doi.org/10.5772/intechopen.88342*

The image is filtered using the image processing techniques (black and white, Gaussian blur, canny edge detection) in order to make the image as simple as possible and have only the lines as a defining feature; this means there is less chance of errors in recognising and classifying the image via the NN.

The images are scaled down to 10x10 in order to make an efficient and enough neural network; the larger the image, the larger the size of the neural network will need to be, which is directly proportionate to the efficiency/iterations per second of the image classifying loop that the device can perform. Thus, a smaller image resolution was chosen in order to make the device as efficient as possible.

Firstly, the filtered image is retrieved from the live stream from the camera and reshaped row by row to create a 1x100 image and converted to a one-dimensional array of 100 values ranging from 0 to 255 to represent each pixel of the image by each particular element of the array. The one-dimensional array is then directly fed into the neural network as the correct input format. Provided this process occurs on each data item, it can then be fed into the network for training, testing and prediction purposes.

#### **4.2 Training the network**

*Control Theory in Engineering*

The second, third and fourth layers have 50, 25 and 12 nodes, respectively; these

The fifth layer which happens to be the output layer has three nodes, which all

The training images are taken by a small script on the Pi which allows a photo to be taken every X seconds. This was set to 0.8. The image resizing and image filters were added and then implemented; while this was running, the car was moved round the track allowing for multiple angles and directions. All of these photos were

From that set of photos, manual collation was required to store each picture in

Once these images had been correctly stored, another part of the program could read them from their containing class folder and compute the following to store

This data is all then stored in Training\_data.xml. The same process occurs again

This process occurs to convert the real-time image into a format that is suitable

its corresponding class folder, in this case, one of 0, 1, or 2.

them in an XML file named Train\_data.xml.

for a neural network's input Layer (**Figure 11**).

for generating/gathering of training data.

*The process of generating training and test data sets.*

are known as hidden layers, and these sizes are calculated automatically.

correspond to the steering control instructions:

• Output 0: FORWARDS | LEFT

• Output 1: FORWARDS | RIGHT

• Output 2: FORWARD ONLY

*4.1.2 Data collection*

**Figure 10.** *System topology.*

then stored on the Pi.

**314**

**Figure 11.**

Training the network is done via the Train() function within OpenCV. Prior to calling this, the associated parameters have to be set. The process of doing this is relatively simple due to the helpful documentation provided in the OpenCV source codes and thus drastically reduces the amount of code required to show this.

Firstly, all values from training data set need to be loaded and added to the stack in their respective math formats (**Figure 12**).

Following this, the network needs to be defined and its topology declared (**Figure 13**).

Next, the relevant network parameters need to be set.

Firstly, the activation function must be set, due to the fact that this network is a Multilayer-perceptron, the chosen activation function is sigmoid (**Figure 14**).

**Figure 12.** *Loading the data from XML file.*

**Figure 13.**

*Declaring the network and assigning its topology.*

**Figure 14.**

*Set the activation function to sigmoid.*

Next, setting the training method and training criteria, a backpropagation rate of 0.0001 for 50,000 iterations was chosen, the higher level of iterations will make the weighting more secure and could be potentially viewed as "over-training"; however, in the given circumstance of a real-time application, the network has to be as accurate as possible. The rate of backpropagation ratio being set to 0.0001 is the standard rate which is used by the majority of users (**Figure 15**).

Following this, all that is required is to call the TRAIN function from the CV library, to run the training algorithm with the above specified topology, parameters and data (**Figure 16**).

#### **4.3 Prediction**

Prediction is used in the testing process and the self-driving process. Prediction is the processing of unknown data in order to categorise the class of the image it is being given.

In the case of testing, this is used to compare the network's output to the correct output in order to calculate the accuracy of the network.

In the self-driving process, the unknown data (camera image and sensor date) is fed into the neural network via prediction to determine the classification of the image. The value returned is used to determine the current state of the track in front of the vehicle, and this is the direction the car should move in.

In the real-time processing part of the project, the real-time image has the aforementioned image preprocessing functions applied to it, to transform it from a fully coloured image into a one-dimensional array of binary values, to match the input layer of the MLP.

Prediction works by receiving this correctly formatted array and passing it to the input layer. This is then fed through the network (which has been fully loaded from the NNVALUES.XML file).

Following this, the network will produce an array of float values, representing the probability of the image belonging to that class. So, in this case it will produce an array of three float values, representing the probability the image which belongs to each of those classified outputs (0, 1, 2).

The highest value in this array is recorded, and its location in the array (0, 1, 2) defines which class the image is most likely to belong to.

This highest value location is then fed into a switch statement which implements a move instruction based on the value.

#### **4.4 Autonomous driving**

Prior to the calling of the AutoDrive function, the following criteria must be met in order for it to work fully:

All hardware devices must be fully installed and working. Training and testing data must be created and stored.

#### **Figure 15.**

*Set training method and training criteria.*

**317**

**Table 1.**

*Unit testing and results.*

*Computational Efficiency: Can Something as Small as a Raspberry Pi Complete…*

Training must be called to train the network and save it once trained. Testing of the network must prove a high accuracy level of the network.

All of this is however done by the administrator to initialise the system prior to

Following this, the device can be placed onto the track and will drive around it, utilising object detection and NN prediction in order to make informed decisions on

It can begin the self-driving Loop in which it will continue driving until the process is halted. This loop will continue to iterate, driving the car in the desired direction based on the current image of the track in front of it, which will change

Unit testing is the testing for the initial components and all basic instructions in the system. These are developed from the basic implementation of the system

Integration testing is testing to see if the more advanced functionalities of the

**Test Test criteria Where Qualifier Pass** Does front motor turn right? TestMotors() Visual Yes Does front motor turn left? TestMotors() Visual Yes Does rear motor go forward? TestMotors() Visual Yes

TestMotors() Visual Yes

TestMotors() Visual Yes

TestSensors() Visual Yes

TestSensors() Visual Yes

DistanceSenseA() Visual Yes

DistanceSenseA() Visual Yes

DistanceSenseB() Visual Yes

DistanceSenseB() Visual Yes

system are comprised from the design process (**Table 2**).

4 Does rear motor go forwards and front motor go

5 Does rear motor go forwards and front motor go

6 Does sensor A function halt when object is within

7 Does sensor B function halt when object is within

8 Sensor A distance of 15 cm shown when object is

9 Sensor A distance of 50 cm shown when object is

10 Sensor B distance of 15 cm shown when object is

11 Sensor B distance of 50 cm shown when object is

*DOI: http://dx.doi.org/10.5772/intechopen.88342*

The track must be ready to use.

demonstration.

each iteration.

**5. Testing**

(**Table 1**).

**5.1 Unit testing**

**5.2 Integration testing**

left?

right?

15 cm?

15 cm?

15 cm away

50 cm away

15 cm away

50 cm away

which directions to drive.

**Figure 16.** *Train model.* *Computational Efficiency: Can Something as Small as a Raspberry Pi Complete… DOI: http://dx.doi.org/10.5772/intechopen.88342*

Training must be called to train the network and save it once trained. Testing of the network must prove a high accuracy level of the network. The track must be ready to use.

All of this is however done by the administrator to initialise the system prior to demonstration.

Following this, the device can be placed onto the track and will drive around it, utilising object detection and NN prediction in order to make informed decisions on which directions to drive.

It can begin the self-driving Loop in which it will continue driving until the process is halted. This loop will continue to iterate, driving the car in the desired direction based on the current image of the track in front of it, which will change each iteration.
