**[Backward elimination]**

8 Principal Component Analysis

{**X**∗(*t*)}*t*≥<sup>0</sup> and *<sup>ρ</sup>*˙ is reduced to zero for all *<sup>r</sup>*. We see from the results that v*ε*-PRINCIPALS

In the analysis of data with large numbers of variables, a common objective is to reduce the dimensionality of the data set. PCA is a popular dimension-reducing tool that replaces the variables in the data set by a smaller number of derived variables. However, for example, in PCA of a data set with a large number of variables, the result may not be easy to interpret. One way to give a simple interpretation of principal components is to select a subset of variables that best approximates all the variables. Various variable selection criteria in PCA has been proposed by Jolliffe [Jolliffe, 1972], McCabe [McCabe, 1984], Robert and Escoufier [Robert and Escoufier, 1976], Krzanowski [Krzanowski, 1987]. Al-Kandari et al. [Al-Kandari et al., 2001; Al-Kandari et al., 2005] gave guidelines as to the types of data for which each variable selection criteria is useful. Cadima et al. [Cadima et al., 2004] reported computational experiments carried out with several heuristic algorithms for the optimization problems resulting from the

Tanaka and Mori [Tanaka and Mori, 1997] proposed modified PCA (M.PCA) for deriving principal components which are computed by using only a selected subset of variables but which represent all the variables including those not selected. Since M.PCA includes variable selection procedures in the analysis, its criteria can be used directly to find a reasonable subset of variables. Mori et al. [Mori et al., 1997] extended M.PCA to qualitative data and provided

M.PCA derives principal components which are computed as linear combinations of a subset of variables but which can reproduce all the variables very well. Let **X** be decomposed into an *n* × *q* submatrix **X***V*<sup>1</sup> and an *n* × (*p* − *q*) remaining submatrix **X***V*<sup>2</sup> . Then M.PCA finds *r* linear combinations **Z** = **X***V*1**A**. The matrix **A** consists of the eigenvectors associated with the largest

matrix of eigenvalues. A best subset of *q* variables has the largest value of the proportion *P* =

In order to find a subset of *q* variables, we employ Backward elimination and Forward selection of Mori et al. [Mori et al., 1998; Mori et al., 2006] as cost-saving stepwise selection

 ∑*r <sup>j</sup>*=<sup>1</sup> *<sup>λ</sup>*<sup>2</sup>

<sup>11</sup> + **S**12**S**21) − **DS**11]**A** = 0, (8)

1/2

. Here we use *P* as variable

is the covariance matrix of **X** = (**X***V*<sup>1</sup> , **X***V*<sup>2</sup> ) and **D** is a *q* × *q* diagonal

*<sup>j</sup>* /tr(**S**2)

*r* eigenvalues *λ*<sup>1</sup> ≥ *λ*<sup>2</sup> ≥···≥ *λ<sup>r</sup>* and is obtained by solving the eigenvalue problem:

}*t*≥<sup>0</sup> and Equation (7) holds,

}*t*≥0. See Brezinski and Zaglia

}*t*≥<sup>0</sup> converges to the final value of

}*t*≥<sup>0</sup> converges to th same limit point **<sup>X</sup>**∗(∞) as {**X**∗(*t*)

}.

**5. Variable selection in nonlinear PCA: Modified PCA approach**

we say that {**X**˙ <sup>∗</sup>(*t*)}*t*≥<sup>0</sup> accelerates the convergence of {**X**∗(*t*)

variable selection criteria in PCA found in the above literature.

variable selection procedures, in which the ASL algorithm is utilized.

[(**S**<sup>2</sup>

procedures in which only one variable is removed or added sequentially.

[Brezinski and Zaglia, 1991]. In the experiments, {**X**˙ <sup>∗</sup>(*t*)

accelerates the convergence of {**X**∗(*t*)

**5.1 Formulation of modified PCA**

 **S**<sup>11</sup> **S**<sup>12</sup> **S**<sup>21</sup> **S**<sup>22</sup>

**5.2 Variable selection procedures**

*<sup>j</sup>*=<sup>1</sup> *λj*/tr(**S**) or the *RV*-coefficient *RV* =

where **S** =

selection criteria.

∑*r*

If {**X**˙ <sup>∗</sup>(*t*)
