**5.2 Results and analysis of our CNN\_DNN neural networks**

After 1000 epoch optimization, the GTEL systems achieved almost 98% training accuracy with 96% test accuracy as shown in **Figure 14**. There is a little overfitting after 400 epochs of training, which can be solved by adding more dropout layers and other methods. **Figure 15** displays the cross-entropy loss during the training process. The training set loss is always decreasing; however, the loss of validation set is flat after 200 epochs of training, indicating some overfitting issue after 200 epochs of training of the data.

**Figure 16** shows the confusion matrix to evaluate our model's performance. As shown, for both training and test datasets, the true labels are almost the same with the predicted label by the GTEL CNN model. The performance evolution pattern is shown in **Figure 17**.

**Figure 13.** *The first 10,000 training tracks with various space behaviors.*

**Figure 14.** *Training and validation accuracy during the training process.*

*Game Theoretic Training Enabled Deep Learning Solutions for Rapid Discovery of Satellite… DOI: http://dx.doi.org/10.5772/intechopen.92636*

**Figure 15.** *Training and validation loss during the training process.*

**Figure 16.** *Confusion matrix of training data and testing data.*

For filter-based normalization, changes are grouped normalized by filters, which aims to display the change distributions over iterations for individual filters. For instance, as shown in **Figure 18**, the filter changes in the convolutional layer and dense layers are visualized. The changes are drastic in the first several iterations and become relatively small in the later stages (after 400 epochs) for most of the layers due to learning rate decay as well as the convergence of the GTEL CNN model.

Notice in **Figure 18** (the color maps are shown on the right side of each plot, the whiter the more stable of the training process), there is no constant deep blue color

**Figure 17.** *Overview of validation classes prediction accuracy during training epochs.*

for all training process, indicating the great performance and reasonable training configuration during training of the deep learning neural networks. Understanding the meaning for these weights during training process with further checking is very important in the CNN model design, which leads to more reasonable explanation in the deployment of the model.

In addition, we investigated the filter image correlations in a broad overview. As shown in **Figure 19**, rows and columns represent layers and image classes, respectively. A sequential color scheme is used to encode the number of anomaly

#### *Game Theoretic Training Enabled Deep Learning Solutions for Rapid Discovery of Satellite… DOI: http://dx.doi.org/10.5772/intechopen.92636*

filters to intuitively represent these relationships between satellite behavior labels and anomaly filters. Using a grid-style visualization, interpretability is possible where rows and columns represent layers and classes of behaviors, respectively. The number of rows and columns equal to the number of layers with anomaly filters and classes with anomaly iterations each class, respectively. In **Figure 19**, the darker the color it is, the more anomaly weights and filters appear in that layer, which are related to that class. From this visualization, it is easy to observe that the second fully connected dense layer has the most anomaly weights and filters among all the other layers. Hence, there are some trends for most anomaly weights and filters during training processes, especially the middle layers of the deep convolutional neural networks. In addition, the anomaly class seems to have the same interval in the dataset, indicating more work is needed for these classes and layers to build a better network. It displays a similar trend compared to the results as shown in **Figure 17**.

**Figure 18.**

*The weight changes in filters during iterations. More blue color indicates stronger variation for the filters during iterations.*

**Figure 19.** *The abstract version of correlation view.*

*Game Theoretic Training Enabled Deep Learning Solutions for Rapid Discovery of Satellite… DOI: http://dx.doi.org/10.5772/intechopen.92636*
