**1. Introduction**

262 Principal Component Analysis

Xiang, Gao; Gang, Wang; Yuan, Li; Ji-hu, Ma (2001) Multivariate Statistical Process

Xiang, Gao; Lina, Bai; Jian Jiang, Cui (2008b). The DTW Synchronized MPCA On-line

China, July 3-7, 2007.

July, 2-4, 2008.

Monitoring Based on Synchronization of Batch Trajectories Using DTW. *4th IFAC workshop on On-line fault detection and supervision in the chemical process industries*, pp. 383-387, ISBN 0080436803, ISBN-13: 9780080436807, Jejudo, Korea, June 7-9, 2001. Xiang, Gao & Lina, Bai (2007). Approach of Prediction and Recovery of Multivariate

Trajectories Based on Generalized Correlation Coefficients. *Proceeding of 2007 Chinese Control and Decision Conference*, pp. 120-122, ISBN 978-7-81102-396-1 Wuxi,

Monitoring and Fault Detection Predicted with GCC. *Proceeding of 2008 Chinese Control and Decision Conference*, pp. 551-555, ISBN 978-1-4244-1733-9, Yantai, China,

> Efficient computation and updating of the principal components are crucial in many applications including data compression, exploratory data analysis, visualization, image processing, pattern and image recognition, time series prediction, detecting perfect and reflective symmetry, and dimension detection. The thorough overview over PCA's applications can be found for example in the textbooks by Duda et al. (2001) and Jolliffe (2002).

> Dynamic versions of the PCA applications, i.e., when the point set (population) changes, are of big importance and interest. Efficient solutions of those problems depend heavily on an efficient dynamic computation of the principal components (eigenvectors of the covariance matrix). Dynamic updates of variances in different settings have been studied since the sixties by Chan et al. (1979), Knuth (1998), Pébay (2008), Welford (1962) and West (1979). Pébay (2008) also investigated the dynamic maintenance of covariance matrices.

> The principal components of discrete point sets can be strongly influenced by point clusters (Dimitrov, Knauer, Kriegel & Rote (2009)). To avoid the influence of the distribution of the point set, often continuous sets, especially the convex hull of a point set is considered, which lead to so-called continuous PCA. Computing PCA bounding boxes (Gottschalk et al. (1996), Dimitrov, Holst, Knauer & Kriegel (2009)), or retrieval of 3D-objects (Vrani´c et al. (2001)), are typical applications where continuous PCA are of interest.

> The organization and the main results presented in this chapter are as follows: In Section 2, we present a standard approach of computing principal components of discrete point set in **R***d*. In Section 3, we present closed-form solutions for efficiently updating the principal components of a set of *n* points, when *m* points are added or deleted from the point set. For both operations performed on a discrete point set in **R***d*, we can compute the new principal components in *O*(*m*) time for fixed *d*. This is a significant improvement over the commonly used approach of recomputing the principal components from scratch, which takes *O*(*n* + *m*) time. In Section 4 we consider the computation of the principal components of a dynamic continuous point set. We give closed form-solutions when the point set is a convex polytope **R**3. Solutions for the cases when the point set is the boundary of a convex polytope in **R**<sup>2</sup> or **R**3, or a convex polygon in **R**2, are presented in the appendix. Conclusion and open problems are presented in Section 5.

of Discrete and Continuous Point Sets 3

Computing and Updating Principal Components of Discrete and Continuous Point Sets 265

closed-form solutions are based on the already computed components of Σ. The recent result of Pébay (2008) implies the same solution for additions that will be presented in the sequel.

**Theorem 3.1.** *Let P be a set of n points in* **R***<sup>d</sup> with known covariance matrix* Σ*. Let P*� *be a point set in* **R***d, obtained by adding or deleting m points from P. The principal components of P*� *can be*

*pk*,*<sup>j</sup>* +

*j* )

*j* ) + <sup>1</sup> *n* + *m*

*n* ∑ *k*=1

*n*+*m* ∑ *k*=*n*+1

*<sup>j</sup>* in (4), we obtain:

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>m</sup>*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>i</sup>* <sup>−</sup> *<sup>m</sup>*

*ij*,1 + *σ*� *ij*,2,

(*pk*,*<sup>i</sup>* − *μ*�

(*pk*,*<sup>i</sup>* − *μ*�

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>m</sup>*

*n* + *m*

*<sup>i</sup>* )( *<sup>m</sup>*

*<sup>i</sup>* )(*pk*,*<sup>j</sup>* − *μj*) +

*<sup>i</sup>* )(*pk*,*<sup>j</sup>* <sup>−</sup> *<sup>n</sup>*

*n* ∑ *k*=1

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>j</sup>* <sup>−</sup> *<sup>m</sup>*

. The *j*-th component, *μ*�

*pk*,*<sup>j</sup>* 

*n*+*m* ∑ *k*=*n*+1

*<sup>i</sup>*)(*pk*,*<sup>j</sup>* − *μ*�

*<sup>i</sup>*)(*pk*,*<sup>j</sup>* − *μ*�

*<sup>i</sup>* )(*pk*,*<sup>j</sup>* − *μ<sup>j</sup>* +

(*pk*,*<sup>i</sup>* <sup>−</sup> *<sup>μ</sup>i*)( *<sup>m</sup>*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>m</sup> <sup>j</sup>* ).

*j*

*j*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>j</sup>* <sup>−</sup> *<sup>m</sup>*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>m</sup> j* )

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>j</sup>* <sup>−</sup> *<sup>m</sup>*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>j</sup>* <sup>−</sup> *<sup>m</sup>*

*m*

*n*+*m* ∑ *k*=*n*+1

*ij*, 1 ≤ *i*, *j* ≤ *d*, of the covariance matrix Σ� of *P*� is

*j*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>j</sup>* <sup>+</sup>

<sup>=</sup> *<sup>n</sup>*

(*pk*,*<sup>i</sup>* − *μ*�

. Those

<sup>1</sup> , *<sup>μ</sup><sup>m</sup>*

, 1 ≤ *j* ≤ *d*, of the center of

*m <sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>m</sup> j* .

*<sup>i</sup>*)(*pk*,*<sup>j</sup>* − *μ*�

*j* ).

), (4)

). (5)

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>m</sup> j* )

*<sup>j</sup>* ) +

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>m</sup>*

<sup>2</sup> ,..., *<sup>μ</sup><sup>m</sup> d* ).

closed-form solutions for computing the components of the new covariance matrix Σ�

Let *Pm* <sup>=</sup> {*pn*+1,*pn*+2,...,*pn*+*m*} be a point set with center of gravity *<sup>μ</sup><sup>m</sup>* = (*μ<sup>m</sup>*

 *n* ∑ *k*=1

*<sup>i</sup>*)(*pk*,*<sup>j</sup>* − *μ*�

*<sup>i</sup>*)(*pk*,*<sup>j</sup>* − *μ*�

*σ*� *ij* = *σ*�

The main result of this section is given in the following theorem.

*<sup>d</sup>*) of *P*� is

*pk*,*<sup>j</sup>* <sup>=</sup> <sup>1</sup>

*n* + *m*

(*pk*,*<sup>i</sup>* − *μ*�

(*pk*,*<sup>i</sup>* − *μ*�

*σ*�

*σ*�

(*pk*,*<sup>i</sup>* <sup>−</sup> *<sup>n</sup>*

(*pk*,*<sup>i</sup>* − *μ<sup>i</sup>* +

*ij*,1 <sup>=</sup> <sup>1</sup> *n* + *m*

*ij*,2 <sup>=</sup> <sup>1</sup> *n* + *m*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>i</sup>* <sup>−</sup> *<sup>m</sup>*

*m*

(*pk*,*<sup>i</sup>* <sup>−</sup> *<sup>μ</sup>i*)(*pk*,*<sup>j</sup>* <sup>−</sup> *<sup>μ</sup>j*) + <sup>1</sup>

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>m</sup>*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>m</sup>*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>i</sup>* <sup>−</sup> *<sup>m</sup>*

*<sup>n</sup>* <sup>+</sup> *<sup>m</sup> <sup>μ</sup><sup>i</sup>* <sup>−</sup> *<sup>m</sup>*

*<sup>i</sup>* and *μ*�

*computed in O*(*m*) *time for fixed d.*

We add *Pm* to *P* obtaining new point set *P*�

<sup>2</sup>,..., *μ*�

*n*+*m* ∑ *k*=1

> *n*+*m* ∑ *k*=1

*n* ∑ =1

<sup>1</sup>, *μ*�

<sup>=</sup> <sup>1</sup> *n* + *m*

*Proof.* **Adding points**

gravity *μ*� = (*μ*�

*μ*� *<sup>j</sup>* <sup>=</sup> <sup>1</sup> *n* + *m*

Let

where,

and

*σ*�

*ij*,1 <sup>=</sup> <sup>1</sup>

<sup>=</sup> <sup>1</sup> *n* + *m*

<sup>=</sup> <sup>1</sup> *n* + *m*

The (*i*, *j*)-th component, *σ*�

Plugging-in the values of *μ*�

*n* + *m*

1 *n* + *m*

1 *n* + *m*

*n* ∑ *k*=1

*m* ∑ *k*=1

*n* ∑ *k*=1

*n* ∑ *k*=1

*n* ∑ *k*=1

( *<sup>m</sup>*

( *<sup>m</sup>*

*σ*� *ij* <sup>=</sup> <sup>1</sup> *n* + *m*

For implementation and verification of some the theoretical results presented here, we refer interested readers to Dimitrov, Holst, Knauer & Kriegel (2009) and Dimitrov et al. (2011).
