**3. Data mining and ML techniques in APT detection**

Most of the APT families stay undetected for a long period and use intelligent ways to damage the vulnerable hosts. When a traditional malware executes, most of the events occur sequentially and leave some traces behind. These traces help modern-day intelligent systems like SIEM, IDS, IPS to prevent these attacks. But, when it comes to the case of APTs, they clean the attack traces and also prevent sequence execution of events. Also, APT employs Anti-VM and Anti-debugging techniques for making things harder for the detection systems. The hardness in detecting the APT has made the cyber security enthusiasts draw their attention towards this domain. Some of the important contributions in the research area are mentioned below. A detailed comparison among different detection techniques are illustrated in **Table 1**, respectively.


#### **Table 1.**

*Comparison of different APT detection methods.*

#### **3.1 A novel deep learning stack for APT detection**

Tero et al. [14] proposed a theoretical approach for detecting APT by developing a stack of Deep Learning methods where each layer has a particular task in handling APT events. The authors consider network payload and packet header information as features, and they streamlined the input to the detection stack without any data filtering mechanism. The detection stack is designed sequentially. The initial layers, i.e. layer-1 and layer-2, are used to detect the known attacks and legitimate network traffic from the data flow respectively. Layer-3 of the detection stack employs in identifying the outliers having historical presence. It uses Recurrent Neural Network-Long Short Term Memory (RNN-LSTM) units to confirm whether an outlier has historical occurrence. Layer-4 helps to classify the outliers into four categories, i.e. regular traffic, known attack, predicted attack and unknown outlier using an anomaly detection method named Growing Hierarchical Self-Organising Map (GHSOM). The stack's final layer helps to map the anomalies (i.e. interconnections between the outlier events) using a Graph Database (GDB). The proposed stack model is highly modular and was designed to perform dynamic detection of APT events with a decent detection accuracy. However, this detection system is complex in design and result in higher time complexity when dealing with massive data inputs.
