**4. Performance results**

We present the results on the performance of the ten deep learning models on the dataset we prepared. We also compare the performances of the models. For designing a robust evaluation framework, we execute every model over ten rounds. The average performance of the ten rounds is considered as the overall performance of the model. We use four metrics for evaluation: (i) average RMSE, (ii) the RMSE for different days (i.e., Monday to Friday) of a week, (iii) the time needed for execution of one round, and (iv) the ratio of the RMSE to the response variable's (i.e., *open* value's) mean value. The models are trained on 19500 historical stock records and then tested on 20250 records. The mean value of the response variable, *open*, of the test dataset is 475.70. All experiments are carried on a system with an Intel i7 CPU with a clock frequency in the range 2.60 GHz – 2.56 GHz and 16GB RAM. The time needed to complete one round of execution of each model is recorded in seconds. The models are built using the Python programming language version 3.7.4 and the frameworks TensorFlow 2.3.0 and Keras 2.4.5.

**Table 11** shows the results of the performance of the CNN\_UNIV\_5 model. The model takes, on average, 174.78 seconds to finish its one cycle of execution. For this model, the ratio of RMSE to the mean *open* values is 0.007288. The ratio of the RMSE to the average of the actual *open* values for day1 through day5 are 0.0062, 0.0066,


### **Table 11.**

*The RMSE and the execution time of the CNN\_UNIV\_5 model.*

*Design and Analysis of Robust Deep Learning Models for Stock Price Prediction DOI: http://dx.doi.org/10.5772/intechopen.99982*

0.0073, 0.0078, and 0.0083, respectively. Here, day1 refers to Monday, and day5 is Friday. In all subsequent Tables, we will use the same notations. The RMSE values of the model CNN\_UNIV\_N\_5 plotted on different days in a week are depicted in **Figure 11** as per record no 2 in **Table 11**.

**Table 12** depicts the performance results of the model CNN\_UNIV\_10. The model needs 185.01 seconds on average for one round. The ratio of the RMSE to the average of the *open* values for the model is 0.006967. The ratios of the RMSE to the average *open* values for day1 through day5 for the model are 0.0056, 0.0067, 0.0070, 0.0075, and 0.0080, respectively. **Figure 12** presents the RMSE values for the results of round 7 in **Table 12**.

**Table 13** depicts the performance results of the model CNN\_MULTV\_10. One round of execution of the model requires 202.78 seconds. The model yields a value of 0.009420 for the ratio of the RMSE to the average of the *open* values. The ratios of the RMSE values to the mean of the *open* values for day1 through day5 of a week are

**Figure 11.** *RMSE vs. day plot of CNN\_UNIV\_5 (depicted by tuple#2 in Table 11).*


**Table 12.**

*The RMSE and the execution time of the CNN\_UNIV\_10 model.*

**Figure 12.** *RMSE vs. day plot of CNN\_UNIV\_10 (depicted by tuple#7 in Table 12).*


### **Table 13.** *The RMSE and the execution time of the CNN\_MULTV\_10 model.*

**Figure 13.** *RMSE vs. day plot of CNN\_MULTV\_10 (based on tuple#6 in Table 13).*

0.0085, 0.0089, 0.0095, 0.0100, and 0.0101, respectively. The RMSE values of the model CNN\_MULTV\_N\_10 plotted on different days in a week are depicted in **Figure 13** based on record number 6 of **Table 13**.


*Design and Analysis of Robust Deep Learning Models for Stock Price Prediction DOI: http://dx.doi.org/10.5772/intechopen.99982*

### **Table 14.**

*The RMSE and the execution time of the CNN\_MULTH\_10 model.*

**Table 14** depicts the results of the model CNN\_MULTH\_10. The model needs, on average, 215.07 seconds to execute its one round. The ratio of the RMSE to the average of the *open* values is 0.008100. The ratios of the RMSE to the average *open* value for day1 to day5 are 0.0076, 0.0075, 0.0082, 0.0084, and 0.0088, respectively. The pattern of variations exhibited by the model daily RMSE is shown in **Figure 14** as per record no 4 in **Table 14**.

The results of the *LSTM\_UNIV\_5* model are depicted in **Table 15**. The average time needed to complete one round of the model is 371.62 seconds. The ratio of the RMSE and the average value of the target variable is 0.007770. The RMSE values for day1 to day5 are 0.0067, 0.0071, 0.0074, 0.0081, and 0.0086, respectively. The pattern of variation of the daily RMSE is as per record no 9 in **Table 15** is depicted in **Figure 15**.

**Table 16** exhibits the results of the model *LSTM\_UNIV\_10*. The model yields a value of 0.007380 for the ratio of its RMSE to the average *open* values, while one round of its execution needs 554.47 seconds. The RMSE values for day1 to day5 are 0.0061, 0.0070, 0.0074, 0079, and 0.0083 respectively. The RMSE pattern of the model as per record no 10 in **Table 16** is exhibited in **Figure 16**.

**Figure 14.** *RMSE vs. day plot of CNN\_MULTH\_10 (based on tuple#4 in Table 14).*


### **Table 15.**

*The RMSE and the execution time of the LSTM\_UNIV\_5 model.*

**Figure 15.** *RMSE vs. day plot of LSTM\_UNIV\_5 (depicted by tuple#9 in Table 15).*

**Table 17** shows that the model LSTM\_UNIV\_ED\_10 needs, on average, 307.27 seconds to execute its one round. The average value of the ratio of the RMSE to the average value of the target variable (i.e., the *open* values) for the model is 0.008350. The daily ratio values for day1 to day 5 of the model are, 0.0067, 0.0078, 0.0085, 0.0090, and 0.0095, respectively. **Figure 17** exhibits the pattern of variation of the daily RMSE as per record no 9 in **Table 17**.

**Table 18** shows that the model LSTM\_MULTV\_ED\_10, on average, requires 634.34 seconds to complete the execution of its one round. For this model, the ratio of the RMSE to the average value of the target variable (i.e., the *open* values) is 0.010294. The ratios of the daily RMSE to the mean value of *open* for day1 to day5 are, respectively, 0.0094, 0.0099, 0.0102, 0.0107, and 0.0111. **Figure 18** shows the pattern of the daily RMSE values of the model as per record no 10 in **Table 18**.

**Table 19** depicts that the model LSTM\_UNIV\_CNN\_N\_10 requires, on average, 222.48 seconds to finish one round. For this model, the ratio of the RMSE to the average value of the target variable (i.e., the *open* values) is found to be 0.007916. The daily RMSE values for day1 to day5 are, 0.0065, 0.0074, 0.0080, 0.0085, and 0.0089



### **Table 16.**

*The RMSE and the execution time of the LSTM\_UNIV\_10 model.*

### **Figure 16.**

respectively. **Figure 19** depicts the pattern of variation of the daily RMSE values for this model as per record no 3 in **Table 19**.

The results of the model LSTM\_UNIV\_CONV\_N\_10 are presented in **Table 20**. The model completes its one round, on average, in 265.97 seconds. The ratio of the RMSE to the average of the *open* values is 0.007490. The daily RMSE for day1 to day5 are 0.0056, 0.0068, 0.0077, 0.0082, and 0.0088, respectively. **Figure 20** shows the patterns of daily RMSE values for this model as per record no 8 in **Table 20**.

**Table 21** summarizes the performance of the ten models proposed in this chapter. We evaluate the models on two metrics and then rank the models on the basis of each metric. The two metrics used for the model evaluation are: (i) an accuracy matric computed as the ratio of the RMSE to the mean value of the target variable (i.e., *open* values), and (ii) a speed metric as measured by the time (in seconds) required for execution of one round of the model. The number of parameters in each model is also presented. It is noted that the CNN\_UNIV\_5 model is ranked 1 on its execution speed, while it occupies rank 2 on the accuracy parameter. The CNN\_UNIV\_10 model, on the other hand, is ranked 2 in terms of its speed of execution, while it is the most accurate model. It is also interesting to note that all the CNN models are faster than their LSTM

*RMSE vs. day plot of LSTM\_UNIV\_10 (depicted by tuple#10 in Table 16).*


**Table 17.** *The RMSE and the execution time of the LSTM\_UNIV\_ED\_10 model.*

**Figure 17.** *RMSE vs. day plot of LSTM\_UNIV\_ED\_10 (as per tuple#5 in Table 17).*

counterparts. However, there is no appreciable difference in their accuracies except for the multivariate encoder-decoder LSTM model, LSTM\_MULTV\_ED\_10.

Another interesting observation is that the multivariate models are found to be inferior to the corresponding univariate models on the basis of the accuracy metric. The multivariate models, CNN\_MULTV\_10 and LSTM\_MULTV\_ED\_10, are ranked 9 and 10, respectively, under the accuracy metric.

Finally, it is observed that the number of parameters in a model has an effect on its execution time, barring some notable exceptions. For the four CNN models, it is noted that with the increase in the number of parameters, there is a monotone increase in the execution time of the models. For the LSTM models, even though the models, LSTM\_UNIV\_CNN\_10, LSTM\_UNIV\_CONV\_10, and LSTM\_UNIV\_ED\_10, have higher number of parameters than the vanilla LSTM models (i.e., LSTM\_UNIV\_5 and LSTM\_UNIV\_10), they are faster in execution. Evidently, the univariate encoderdecoder LSTM models are faster even when they involve a higher number of parameters than the vanilla LSTM models.



### **Table 18.**

*The RMSE and the execution time of the LSTM\_MULTV\_ED\_10 model.*

### **Figure 18.**

*RMSE vs. day plot of LSTM\_MULTV\_ED\_10 (as per tuple#10 in Table 18).*



### **Table 19.**

*The RMSE and the execution time of the LSTM\_UNIV\_CNN\_10 model.*

#### **Figure 19.**

*RMSE vs. day plot of LSTM\_UNIV\_CNN\_10 (as per tuple#3 in Table 19).*


### **Table 20.**

*The RMSE and the execution time of the LSTM\_UNIV\_CONV\_10 model.*

**Figure 20.** *RMSE vs. day plot of LSTM\_UNIV\_CONV\_10 (as per tuple#8 in Table 20).*

*Design and Analysis of Robust Deep Learning Models for Stock Price Prediction DOI: http://dx.doi.org/10.5772/intechopen.99982*


**Table 21.**

*Comparative analysis of the accuracy and execution speed of the models.*
