5.1. Synthetic data sets

local manifold learning method. The global embedding coordinates <sup>Y</sup> ¼ ½y1;y2;…;yN�∈ℝ<sup>d</sup> · <sup>N</sup> can be obtained by minimizing the functional GðfÞ. Let y ¼ f ¼ ½fðx1Þ;…;fðxNÞ� be a row vector of Y. It is not hard to show that the global embedding coordinates and the nonnegative

where the power parameter r > 1 is set to avoid the phenomenon that the solution to c is cj ¼ 1

wise, since our aim is to utilize the complementary geometric information from different

We propose to solve the objective function [Eq. (33)] by employing the alternating optimization [16] method, which iteratively updates Y and c in an alternating fashion. First, we fix c to

obtain the global optimal solution Y as the second to ðd þ 1Þ st smallest eigenvectors of the matrix P. Second, we fix Y to update c. While Y is fixed, we can minimize the objective function [Eq. (33)] analytically through utilizing a Lagrange multiplier to enforce the constraint that

<sup>Y</sup><sup>T</sup>ÞÞ<sup>1</sup>=ðr−1<sup>Þ</sup>

In this section, we experiment on both synthetic and real-world data sets to evaluate the performance of our method, named FLM. For LEM, LLE, HLLE, LTSA, and our Fusion of local manifolds (FLM) algorithms, we experiment on these data sets to obtain both visualization and quantitative evaluations. We utilize the global smoothness and co-directional consistence (GSCD) criteria [17] to quantitatively compare the embedding qualities of different algorithms: the smaller the value of GSCD, the higher the global smoothness, and the better the codirectional consistence. There are two adjustable parameters in our FLM method, that is, the tuning parameter r and the number of nearest neighbors k. FLM works well when the values of r and k are neither too small nor too large. The reason is that only one local method is chosen when r is too small, while the relative weights of different methods tend to be close to each other when it is too large. As a general recommendation, we suggest to work with r∈½2; 6� and

<sup>Y</sup><sup>T</sup><sup>Þ</sup> <sup>s</sup>:t:YY<sup>T</sup> <sup>¼</sup> <sup>I</sup>; <sup>∑</sup>

M j¼1

. When c is fixed, we can solve the optimization problem [Eq. (34)] and

<sup>Y</sup><sup>T</sup><sup>Þ</sup> over different local methods and ck <sup>¼</sup> <sup>0</sup>ðk≠j<sup>Þ</sup> other-

TrðYPY<sup>T</sup><sup>Þ</sup> <sup>s</sup>:t: YY<sup>T</sup> <sup>¼</sup> <sup>I</sup> (34)

<sup>Y</sup><sup>T</sup>ÞÞ<sup>1</sup>=ðr−1<sup>Þ</sup> ; <sup>j</sup> <sup>¼</sup> {1;…;M} (35)

cj ¼ 1; cj≥0: (33)

weights c ¼ ½c1;…;cM� can be obtained by minimizing the following objective function:

argmin Y;c

corresponding to the minimum TrðYPj

manifold learning methods.

142 Manifolds - Current Research Areas

<sup>j</sup>¼<sup>1</sup>cr jPj

5. Experimental results

k∈½0:7⌈logðNÞ⌉, 2⌈logðNÞ⌉�.

where <sup>P</sup> <sup>¼</sup> <sup>∑</sup><sup>M</sup>

∑<sup>M</sup>

∑ M j¼1 c r jTrðYP<sup>j</sup>

update Y. The optimization problem in Eq. (33) is equivalent to:

<sup>j</sup>¼<sup>1</sup>cj <sup>¼</sup> 1. And the global optimal <sup>c</sup> can be obtained as:

cj <sup>¼</sup> <sup>ð</sup>1=TrðYP<sup>j</sup>

<sup>j</sup>¼<sup>1</sup>ð1=TrðYPj

∑<sup>M</sup>

argmin Y

We first apply our FLM to the synthetic data sets that have been commonly used by other researchers: S-Curve, Swiss Hole, Punctured Sphere, and Toroidal Helix. The character of these data sets can be summarized as: general, non-convex, nonuniform, and noise, respectively. In each data set, we have total 1000 sample points, and the number of nearest neighbors is fixed to k ¼ 10 for all the algorithms. For the S-Curve and Swiss Hole, we empirically set r ¼ 2, and for the Punctured Sphere and Toroidal Helix data sets, we set r=3. Figures 2–5 show the embedding results of the above algorithms on the four synthetic data sets. Each manifold learning algorithm and the corresponding GSCD result are shown in the title of each subplot. We can evaluate the performances of these methods by comparing the coloring of the data points, the smoothness, and the shape of the projection coordinates with their original manifolds. Figures 2–5 reveal the following interesting observations.


The above consequence is because only partial geometric information of the underlying manifold is learned by each traditional local manifold learning method, while the complementary geometric information learned from different manifold learning algorithms is respected by our FLM method.
