3.3 Machine learning models

3. Supervised feature extraction

Time Series Analysis - Data, Methods, and Applications

Autocorrelated data can be addressed by fitting models to the data and analyzing

the residuals, instead of the variables. With ARIMA models, crosscorrelation between the variables is not accounted for, and although multivariate models can also be employed using this approach, it becomes a complex task when there are many variables (m > 10), owing to the high number of parameters that must be

Apart from ARIMA models, other models, such as neural networks [60–62], decision trees [63], and just-in-time-learning with PCA [64], have also been proposed.

If it is assumed that the data matrix X contains all the dynamic information of

x<sup>k</sup>þ<sup>1</sup> ¼ fð Þþ x<sup>k</sup> w<sup>k</sup> (6) y<sup>k</sup> ¼ gð Þþ x<sup>k</sup> v<sup>k</sup> (7)

> Partial PCA [12, 13] Kernel PCA [80] Multiscale [81]

Kernel ICA [88]

Kernel [88]

Complex networks [52]

Recurrence quantification

Attractor descriptors [49–51, 90]

Deep learning [76, 77, 96]

[2, 53–58]

77, 93–95]

Dynamic PCA Linear PCA [4, 8, 78, 79]

Dynamic ICA ICA [14, 82–87]

Slow feature analysis [22–24] ICA Standard [89]

analysis

Dissimilarity [84, 91, 92]

Autoregressive models [59, 71, 93] State space models [65–70, 76,

Machine learning Conventional [2, 48, 60–62]

the system, then the use of predictive models can be viewed as an attempt to remove all the dynamic information from the system to yield Gaussian residuals that can be monitored in the normal way. State space models offer a principled approach for the identification of the subspaces containing the data. This can be

Class Method References

Phase space and related

methods

Approaches to the monitoring of continuous dynamic process systems.

estimated, as well as the presence of crosscorrelation [3, 59].

3.1 Autoregressive models

3.2 State space models

summarized as follows

Unsupervised feature

Supervised feature extraction

Table 2.

10

extraction

In principle, machine learning models are better able to deal with complex nonlinear systems than linear models, and some authors have considered the use of these approaches. For example, Chen and Liao [62] have used a multilayer perceptron neural network to remove the nonlinear and dynamic characteristics of processes to generate residuals that could be used as input to a PCA model for the construction of simple monitoring charts. Guh and Shiue [63] have used a decision tree to detect shifts in the multivariate means of process data. Auret and Aldrich [48] have considered the use of random forests in the detection of change points in process systems. In addition, Aldrich and Auret [2] have compared the use of random forests with autoassociative neural networks and singular spectrum analysis in a conventional process monitoring framework.

The application of deep learning in process monitoring is an emerging area of research that shows particular promising. This includes the use of stacked autoencoders [76], deep long short term memory (LSTM) neural networks [77], and convolutional neural networks. Table 2 gives an overview of the feature extraction methods that have been investigated over the last few decades.
