**Application of Linear and Nonlinear Dimensionality Reduction Methods**

Ramana Vinjamuri1,4, Wei Wang1,4, Mingui Sun2 and Zhi-Hong Mao<sup>3</sup>

*Department of Physical Medicine and Rehabilitation Department of Neurological Surgery Department of Electrical and Computer Engineering Center for Neural Basis of Cognition University of Pittsburgh, Pittsburgh, PA USA*

#### **1. Introduction**

16 Will-be-set-by-IN-TECH

106 Principal Component Analysis

Belhumeur, P. N., Hepanha, J. P. & Kriegman, D. J. (1997). Eigenfaces vs. fisherfaces:

Cai, D. (2009). Spectral regression: A regression framework for efficient regularized subspace

Cai, D., He, X., Hu, Y., Han, J. & Huang, T. (2007a). Learning a spatially smooth subspace for

Cai, D., He, X., Hu, Y., Han, J. & Huang, T. (2007b). Spectral regression for efficient regularized

Casilescu, M. & D.Terzopoulos (2002). Multilinear analysis of image ensembles: Tensorfaces,

Han, X.-H., Qiao, X. & wei Chen, Y. (2011). Multilinear supervised neighborhood embedding with local descriptor tensor for face recognition, *IEICE Trans. Inf.* & *Syst.* . Han, X.-H., wei Chen, Y. & Ruan, X. (2011). Multilinear supervised neighborhood embedding

Harris, C. & Stephens, M. (1998). A combined corner and edge detector, *In Proc. Alvey Vision*

He, X., Yan, S., Hu, Y., Niyogi, P. & Zhang, H.-J. (2005). Face recognition using laplacianfaces, *IEEE Transactions on Pattern Analysis and Machine Intelligence* (3): 328–340. Lazebnik, S., Schmid, C. & Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching

Leibe, B. & Schiele, B. (2003a). Analyzing appearance and contour based methods for object

Leibe, B. & Schiele, B. (2003b). Analyzing appearance and contour based methods for object

Lowe, D. (2004). Distinctive image features from scale-invariant keypoints, *International*

Marrr, R., Geurts, P., Piater, J. & Wehenkel, L. (2005). Random subwindows for robust image

ming Wang, X., Huang, C., ying Fang, X. & gao Liu, J. (2009). 2dpca vs. 2dlda: Face recognition

Nene, S. A., Nayar, S. K. & Murase, H. (1996). Columbia object image library (coil-100),

Pontil, M. & Verri, A. (1998). Support vector machines for 3d object recognition, *PAMI*

Sim, T., Baker, S. & Bsat, M. (2001). The cmu pose, illumination, and expression (pie) database of human faces, *Robotics Institute, CMU-RI-TR-01-02, Pittsburgh, PA,* . Wang, Y. & Gong, S. (2006). Tensor discriminant analysis for view-based object recognition,

L.D. Lathauwer. Signal processing based on multilinear algebra, *Ph.D. Thesis*, Katholike

using two-dimensional method, *International Conference on Artificial Intelligence and*

for recognizing natural scene categories, *CVPR* pp. 2169–2178.

of local descriptor tensor for scene/object recognition, *IEEE Transaction on Image*

*and Machine Intelligence* (7): 711–720.

face recognition, *CVPR* .

subspace learning, *ICCV* .

Cai, D. (n.d.). *http://www.zjucadcg.cn/dengcai/Data/Yale/results\_new.html* .

recognition using class specific linear projection, *IEEE Transactions on Pattern Analysis*

**8. References**

learning.

*ECCV* .

*Processing* .

*Conference* .

categorization, *CVPR* .

categorization, *CVPR* .

*Journal of Computer Vision* (2): 91–110.

*Computational Intelligence* pp. 357–360.

classification, *CVPR* pp. 34–40.

*Technical Report CUCS-006-96* .

pp. 637–646.

*ICPR* pp. 439–454.

Universiteit Leu- ven.

Dimensionality reduction methods have proved to be important tools in exploratory analysis as well as confirmatory analysis for data mining in various fields of science and technology. Where ever applications involve reducing to fewer dimensions, feature selection, pattern recognition, clustering, dimensionality reduction methods have been used to overcome the curse of dimensionality. In particular, Principal Component Analysis (PCA) is widely used and accepted linear dimensionality reduction method which has achieved successful results in various biological and industrial applications, while demanding less computational power. On the other hand, several nonlinear dimensionality reduction methods such as kernel PCA (kPCA), Isomap and local linear embedding (LLE) have been developed. It has been observed that nonlinear methods proved to be effective only for specific datasets and failed to generalize over real world data, even at the cost of heavy computational burden to accommodate nonlinearity.

We have systematically investigated the use of linear dimensionality reduction methods in extracting movement primitives or synergies in hand movements in Vinjamuri et al. (2010a;b; 2011). In this chapter, we applied linear (PCA and Multidimensional Scaling (MDS)) and nonlinear (kPCA, Isomap, LLE) dimensionality reduction methods in extracting kinematic synergies in grasping tasks of the human hand. At first, we used PCA and MDS on joint angular velocities of the human hand, to derive synergies. The results obtained indicated ease and effectiveness of using PCA. Then we used nonlinear dimensionality reduction methods for deriving synergies. The synergies extracted from both linear and nonlinear methods were used to reconstruct the joint angular velocities of natural movements and ASL postural movements by using an *l*1-minimization algorithm. The results suggest that PCA outperformed all three nonlinear methods in reconstructing the movements.

#### **2. Synergies**

The concept of synergies (in Greek *synergos* means working together) was first represented numerically by Bernstein Bernstein (1967). Although synergies were originally defined by Bernstein as high-level control of kinematic parameters, different definitions of synergies exist

**s**2

from Vinjamuri et al. (2011)

synergy-based neural prosthesis.

the execution of movements.

Angular

Fig. 1. Two distinct synergies (**s**1, **s**2) use a weighted linear combination (*w*1**s**<sup>1</sup> + *w*2**s**2) to achieve a grasping hand movement. *w*<sup>1</sup> and *w*<sup>2</sup> represent weights of control signals. Each row of a synergy corresponds to the angular velocity profile of a finger joint; For example, the first synergy represents the synchronous large movement of first joint and medium movement of the second joint followed by a small movement of the third joint. In this example, **s**<sup>1</sup> (blue) and **s**<sup>2</sup> (brown) form a weighted (*w*<sup>1</sup> = *w*<sup>2</sup> = 0.5) combination to result in the aggregate movement(black) on the right hand side. For illustration purposes, only 3 of 10 joints of the hand are shown in the synergies and the reconstructed movement. Also shown in the figure are the hand postures of the reconstructed movement across time. Adapted

on the principles of data reduction and dimensionality reduction, are soon to find place in tele-surgery and tele-robotics ( Vinjamuri et al. (2007)). Synergies are projected to be miniature windows to provide immense help in next generation rehabilitation. Recently our group has demonstrated a synergy based brain machine interface where two control signals calculated from the spectral powers of the brain signals controlled two synergies, that commanded a 10 DoF virtual hand ( Vinjamuri et al. (2011)). This showed promising results for controlling a

(ii) Diagnostics: Applying similar concepts of synergies on the hand movements of the individuals with movement disorders, the sources that contain the tremor were isolated. Using blind source separation and dimensionality reduction methods, the possible neural sources that contained tremor were extracted from the hand movements of individuals with Essential Tremor ( Vinjamuri et al. (2009)). This led to an efficient quantification of tremor. (iii) Robotics: Biologically inspired synergies are being used in balance control of humanoid robots ( Hauser et al. (2007)). Based on the principle that biological organisms recruit kinematic synergies that manage several joints, a control strategy for balance of humanoid robots was developed. This control strategy reduced computational complexity following a biological framework that central nervous system reduces the computational complexity of managing numerous degrees of freedom by effectively utilizing the synergies. Biologically inspired neural network controller models ( Bernabucci et al. (2007)) that can manage ballistic arm movements have been developed. The models simulated the kinematic aspects, with bell-shaped wrist velocity profiles, and generated movement specific muscular synergies for

(iv) Rehabilitation: Bimanual coordination is damaged in brain lesions and brain disorders Vinjamuri et al. (2008). Using a small set of modifiable and adjustable synergies

Time

Reconstructed joint angular velocities

velocities of 3

 joints

Application of Linear and Nonlinear Dimensionality Reduction Methods 109

**s**1

Synergies

and the term has been generalized to indicate the shared patterns observed in the behaviors of muscles, joints, forces, actions, etc. Synergies in hand movements especially present a complex optimization problem as to how the central nervous system (CNS) controls the hand with over 25 degrees of freedom(DoF) ( Mackenzie & Iberall (1994)). Yet, the CNS handles all the movements effortlessly and at the same time dexterously. Endeavoring to solve the DoF problem, many researchers have proposed several concepts of synergies such as the following:

(i) Postural synergies: In Jerde et al. (2003); Mason et al. (2001); Santello et al. (1998; 2002); Thakur et al. (2008); Todorov & Ghahramani (2004), researchers found that the entire act of grasp can be described by a small number of dominant postures, which were defined as postural synergies.

(ii) Kinematic synergies: Studies in Grinyagin et al. (2005); Vinjamuri et al. (2007) expressed the angular velocities of finger joints as linear combinations of a small number of kinematic synergies, which were also angular velocities of finger joints but were extracted from a large set of natural movements. Kinematic synergies are not limited to hand movements. In ( d'Avella et al. (2006)), d'Avella et al. reported that kinematic synergies were found in tracking 7 DoF arm movements.

(iii) Dynamic synergies: Dynamic synergies were defined as stable correlations between joint torques that were found during precision grip movements in Grinyagin et al. (2005).

The above classification was already presented in Vinjamuri et al. (2010b). In addition to synergies proposed in postures, and kinematics which are of relevance to the current study, synergies were also proposed in muscle activities d'Avella et al. (2006).
