3.2. Leading EVs and SchVs as optimal perturbations

Theorem 2.1. The optimal perturbation for the matrix Φ<sup>i</sup>�<sup>1</sup>, i ¼ 1, …, n is ð Þ þ=� vi where vi is

Comment 2.2. The OPs, presented above, are optimal in the sense of the Euclidean norm k k: <sup>2</sup>. In practice, there is a need to normalize the state vector (using the inverse of the covariance

As to the PE, a normalization is also applied in order to have a possibility to compare different variables like density, temperature, velocity … In this situation, the norm for y ¼ Φδx may be

The idea of ensemble forecasting is that instead of performing "deterministic" forecasts, stochastic forecasts should be made: several model forecasts are performed by introducing per-

Since 1994, NCEP (National Centers for Environmental Prediction, USA) has been running 17 global forecasts per day, with the perturbations obtained using the method of breeding growing perturbations. This ensures that the perturbations contain growing dynamical perturbations. The length of the forecasts allows the generation of outlook for the second week. At the ECMWF, the perturbation method is based on the use of SVs, which grow even faster than the bred or Lyapunov vector perturbations. The ECMWF ensemble contains 50

The idea underlying the AF is to construct a filter which uses feedback in the form of the PE signal (innovation) to adjust the free parameters in the gain to optimize the filter performance. If in the KF, the optimality is defined as a minimum mean squared error (MMS), in the AF, optimality is understood in the sense of MMS for the prediction output error (innovation). This definition allows to define the optimality of the filter in the realization space, but not in the

The optimal gain thus can be determined from solving the optimization problem by adjusting

<sup>δ</sup><sup>x</sup> <sup>&</sup>gt;¼<sup>&</sup>lt; <sup>δ</sup>x, M�<sup>1</sup>

th SOP—singular OP).

δx > ≔ k k δx <sup>M</sup>�<sup>1</sup>

δx, and all the results presented

th OP for Φ (or the i

the ith leading right SV of Φ<sup>i</sup>�<sup>1</sup>, i ¼ 1, …, n.

66 Perturbation Methods with Applications in Science and Engineering

matrix <sup>M</sup>). The normalization is done by changing <sup>δ</sup>x<sup>0</sup> <sup>¼</sup> <sup>M</sup>�1=<sup>2</sup>

The weighted norm k k δx <sup>M</sup>�<sup>1</sup> is known as the Mahanalobis norm.

<sup>δ</sup>x<sup>0</sup> k k<sup>2</sup> <sup>¼</sup><sup>&</sup>lt; <sup>M</sup>�1=<sup>2</sup>

turbations in the filtered estimate or in the models.

3. Perturbations based on leading EVs and SchVs

all elements of the filter gain. There are two major difficulties:

,

δx, M�1=<sup>2</sup>

The OP for Φ<sup>i</sup>�<sup>1</sup> will be called the i

above remain valid s.t. the new δx<sup>0</sup>

seminorm [5].

members [13].

3.1. Adaptive filter (AF)

probability space as done in the KF.

2.4. Ensemble forecasting

Interest on stability of the AF arises soon after the AF has been introduced. The study on the filter stability shows that it is possible to provide a filter stability when the system is detectable [8]. For the different parameterized stabilizing gain structures based on a subspace of unstable and neutral EVs, see [8]. As the EVs may be complex and their computation is unstable (Lanczos [14]), in [8], it is proved that one can also ensure a stability of the filter if the space of projection is constructed from a set of unstable and neutral SchVs of the system dynamics. The unstable and neutral real SchVs are referred to as SchVs associated with the unstable and neutral eigenvalues of the system dynamics. The advantage of the real SchVs is that they are real, orthonormal, and their computation is stable. Moreover, the algorithm for estimating dominant SchVs is simple which is based on the power iteration procedure (Sampling-P, see [15]). As to the unstable SVs, although they are real and orthonormal, their computation requires an adjoint operator (the transpose matrix Φ<sup>T</sup>). Construction of adjoint code (AC) is a time-consuming and tedious process. Approximating leading SVs without (AC) can be done on the basis of Algorithms 5.2.

#### 3.3. EVs as optimal perturbations in the invariant subspace

Let Φ be diagonalizable. Introduce the set

$$\{EV\_1(\mathbf{x}, \lambda) : \mathbf{x} \in \mathbb{C}^n, \|\mathbf{x}\|\_2 = 1, \lambda \in \mathbb{C}^1 : \Phi \mathbf{x} = \lambda \mathbf{x}\}.\tag{11}$$

The subspace of <sup>x</sup><sup>∈</sup> Cn satisfying <sup>Φ</sup><sup>x</sup> <sup>¼</sup> <sup>λ</sup><sup>x</sup> for some <sup>λ</sup> <sup>∈</sup>C<sup>1</sup> is known as an invariant subspace of Φ: the matrix Φx acts on to stretch the vector x but conserves the direction of x. Consider the optimization problem

$$J(\delta \mathbf{x}) = \begin{array}{l} \|\Phi \delta \mathbf{x}\|\_{2} \to \max\_{\delta \mathbf{x}} \\ (\delta \mathbf{x}, \lambda) \in EV\_{1}(\delta \mathbf{x}, \lambda), \end{array} \tag{12}$$

It is seen that the optimal solution is the first EV xeið Þ1 of Φ with the largest magnitude equal to ∣λ1∣. We will call λ<sup>1</sup> a first optimal EV perturbation (denoted as EI-OP).

For a symmetric matrix, the EI-OP coincides with the SOP. The EI-OP is not unique.

By solving the optimization problem (8) s.t.

$$\{EV\_2(\mathbf{x}, \boldsymbol{\lambda}) : \mathbf{x} \in \mathbb{C}^n, \|\mathbf{x}\|\_2 = 1 : \Phi \mathbf{x} = \boldsymbol{\lambda} \mathbf{x}, \boldsymbol{\lambda} \in \mathbb{C}^1, |\boldsymbol{\lambda}| < |\boldsymbol{\lambda}\_1|\}. \tag{13}$$

one finds the second EI-OP xeið Þ2 . In a similar way, by defining

$$\{EV\_i(\mathbf{x}, \boldsymbol{\lambda}) : \mathbf{x} \in \mathbb{C}^n, \|\mathbf{x}\|\_2 = 1 : \Phi \mathbf{x} = \lambda \mathbf{x}, \lambda \in \mathbb{C}^1, |\lambda| < |\lambda\_{i-1}|\}. \tag{14}$$

for i ¼ 1, 2, ::, n � 1, we obtain a sequence of EI-OPs xeið Þi , i ¼ 1, 2, ::, n. The first ne EI-OPs are unstable SVs.

In general, for a defective case (not diagonalizable), Φ does not have n linearly independent EVs and the independent generalized EVs can serve as "optimal" perturbations to construct a subspace of projection in the AF.

To summarize, let the EV decomposition be

$$X\_{\rm ef} \!\!\!X\_{\rm ef}^{-1} = \Phi \tag{15}$$

random vector (RV) with the ECM P. The question arising here is how one determine the OP in

On Optimal and Simultaneous Stochastic Perturbations with Application to Estimation of High-Dimensional…

We will consider now δx as an element of the Hilbert space H of RVs. This space H is a

k k<sup>x</sup> <sup>H</sup> <sup>¼</sup> ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi

All elements of H are of finite variance and for simplicity, we assume they all have zero mean

Definition 4.1. The optimal stochastic perturbation (OSP) δxo is the solution of the extremal

<sup>δ</sup>xo <sup>¼</sup> <sup>ψ</sup>v<sup>1</sup>

Comment 4.1. Comparing the ODP with OSP shows that if the ODS is the first right SV (defined up to the sign), the OSP is an ensemble of vectors lying in the subspace of the first right SV with

<sup>Φ</sup><sup>1</sup> <sup>≔</sup> <sup>Φ</sup> � <sup>σ</sup>1u1vT

Lemma 4.3. The optimal perturbation in the sense (21) and (19) is <sup>δ</sup>xo <sup>¼</sup> <sup>ψ</sup>v2, where <sup>ψ</sup> is an RV

1

Jð Þ¼ δx k k Φ1δx <sup>H</sup> ! maxδx, (21)

� �<sup>T</sup>

< x, y><sup>H</sup> ¼ E < x, y > (17)

Ssð Þ <sup>δ</sup><sup>x</sup> <sup>≔</sup> <sup>δ</sup><sup>x</sup> : k k <sup>δ</sup><sup>x</sup> <sup>H</sup> <sup>¼</sup> <sup>1</sup> � � (19)

Jð Þ¼ δx k k Φδx <sup>H</sup> ! maxδx, (20)

<sup>∈</sup> Ssð Þ <sup>δ</sup><sup>x</sup> such that <sup>δ</sup><sup>x</sup> <sup>¼</sup> <sup>P</sup><sup>n</sup>

<sup>k</sup>¼<sup>1</sup> vkδyk.

E < x, x > p (18)

http://dx.doi.org/10.5772/intechopen.77273

69

complete normed linear vector space equipped with the inner product

where Eð Þ: denotes the mathematical expectation. The norm in H is defined as

such situation and how to find it.

value.

problem

Introduce

Introduce the set of RVS

under the constraint (19). One can prove

and consider the objective function

Lemma 4.1. For δx∈ Ssð Þ δx , there exists δy ¼ δy1;…; δyn

Lemma 4.2. The optimal perturbation in the sense (20) and (19) is

where ψ is a RV with zero mean and unit variance, v<sup>1</sup> is the first right SV of Φ.

the lengths being the samples of the RV of zero mean and unit variance.

with zero mean and unit variance, v<sup>2</sup> is the first right SV of Φ.

where J is a matrix of Jordan canonical form, X�<sup>1</sup> ei is the matrix inverse of Xei (see Golub and Van Loan [9]). The columns of Xei are the EVs of Φ, J is a block diagonal with the diagonal blocks of 1 or 2 dimensions. The rank k decomposition is Xei,1J1X~ ei,<sup>1</sup> where

$$\begin{aligned} EV\_k &:= X\_{ci,1} I\_1 \ddot{X}\_{ei,1}, \\ X\_{ei} &= [X\_{ei,1}, X\_{ei,2}]\_\prime J = \text{block diag} [I\_1, I\_2]\_\prime \\ \tilde{X}\_{ei} &:= X\_{ei}^{-1} = \left[ \tilde{X}\_{ei,1}^T, \tilde{X}\_{ei,2}^T \right]\_\prime \end{aligned} \tag{16}$$

with Xei,<sup>1</sup> ∈R<sup>n</sup>�<sup>k</sup> , Xei, <sup>2</sup> ∈ Rn�ð Þ <sup>n</sup>�<sup>k</sup> . Multiplying the right of EVk by Xei,<sup>1</sup> yields Xei, <sup>1</sup>J1, i.e., we obtain the k largest (in modulus) perturbations in the eigen (invariant) space of Φ. The perturbations being the column vectors of Xei,<sup>1</sup> (i.e., the k first EVs of Φ) are the first k OPs of Φ in the eigen-invariant subspace (EI-InS).

#### 3.4. Dominant SchVs as OPs in the Schur invariant subspace

The study of Hoang et al. [8] shows that the subspace of projection of the stable filter can be constructed on the basis of all unstable EVs or SchVs of Φ.

Compared to the EVs, the approach based on real Schur decomposition is of preference in practice since the SchVs are real and orthonormal. Moreover, there exists a simple, power iterative algorithm for approaching the set of real leading SchVs. According to Theorem 7.3.1 in Golub and Van Loan [9], the subspace R X½ � s,<sup>1</sup> spanned by the nu leading SchVs converges to the unique invariant subspace Dnu ð Þ Φ (called a dominant invariant subspace) associated with the eigenvalues λ1, …, λnu if ∣λnu ∣ > ∣λnuþ<sup>1</sup>∣. In this sense, we consider the leading SchVs as OPs (denoted as Sch-OP) in the Schur invariant subspace (Sch-InS).

#### 4. Optimal stochastic perturbation (OSP)

In Section 2, the perturbation δx is deterministic (see Definition 2.1). In practice, it happens that δx is of stochastic nature. For example, the priori information on the FE is an zero mean random vector (RV) with the ECM P. The question arising here is how one determine the OP in such situation and how to find it.

We will consider now δx as an element of the Hilbert space H of RVs. This space H is a complete normed linear vector space equipped with the inner product

$$<\mathbf{x}, \mathbf{y} >\_H = E<\mathbf{x}, \mathbf{y} > \tag{17}$$

where Eð Þ: denotes the mathematical expectation. The norm in H is defined as

$$\|\mathbf{x}\|\_{H} = \sqrt{E < \mathbf{x}, \mathbf{x} >} \tag{18}$$

All elements of H are of finite variance and for simplicity, we assume they all have zero mean value.

Introduce the set of RVS

EVið Þ <sup>x</sup>; <sup>λ</sup> : <sup>x</sup> <sup>∈</sup>C<sup>n</sup>; k k<sup>x</sup> <sup>2</sup> <sup>¼</sup> <sup>1</sup> : <sup>Φ</sup><sup>x</sup> <sup>¼</sup> <sup>λ</sup>x; <sup>λ</sup> <sup>∈</sup>C<sup>1</sup>

unstable SVs.

with Xei,<sup>1</sup> ∈R<sup>n</sup>�<sup>k</sup>

eigen-invariant subspace (EI-InS).

subspace of projection in the AF.

To summarize, let the EV decomposition be

where J is a matrix of Jordan canonical form, X�<sup>1</sup>

68 Perturbation Methods with Applications in Science and Engineering

for i ¼ 1, 2, ::, n � 1, we obtain a sequence of EI-OPs xeið Þi , i ¼ 1, 2, ::, n. The first ne EI-OPs are

In general, for a defective case (not diagonalizable), Φ does not have n linearly independent EVs and the independent generalized EVs can serve as "optimal" perturbations to construct a

XeiJX�<sup>1</sup>

blocks of 1 or 2 dimensions. The rank k decomposition is Xei,1J1X~ ei,<sup>1</sup> where

ei <sup>¼</sup> <sup>X</sup><sup>~</sup> <sup>T</sup>

EVk ≔ Xei, <sup>1</sup>J1X~ ei, <sup>1</sup>,

X~ ei ≔ X�<sup>1</sup>

3.4. Dominant SchVs as OPs in the Schur invariant subspace

(denoted as Sch-OP) in the Schur invariant subspace (Sch-InS).

4. Optimal stochastic perturbation (OSP)

constructed on the basis of all unstable EVs or SchVs of Φ.

Van Loan [9]). The columns of Xei are the EVs of Φ, J is a block diagonal with the diagonal

Xei ¼ ½ � Xei, <sup>1</sup>; Xei, <sup>2</sup> , J ¼ block diag J1; J<sup>2</sup> ½ �,

obtain the k largest (in modulus) perturbations in the eigen (invariant) space of Φ. The perturbations being the column vectors of Xei,<sup>1</sup> (i.e., the k first EVs of Φ) are the first k OPs of Φ in the

The study of Hoang et al. [8] shows that the subspace of projection of the stable filter can be

Compared to the EVs, the approach based on real Schur decomposition is of preference in practice since the SchVs are real and orthonormal. Moreover, there exists a simple, power iterative algorithm for approaching the set of real leading SchVs. According to Theorem 7.3.1 in Golub and Van Loan [9], the subspace R X½ � s,<sup>1</sup> spanned by the nu leading SchVs converges to the unique invariant subspace Dnu ð Þ Φ (called a dominant invariant subspace) associated with the eigenvalues λ1, …, λnu if ∣λnu ∣ > ∣λnuþ<sup>1</sup>∣. In this sense, we consider the leading SchVs as OPs

In Section 2, the perturbation δx is deterministic (see Definition 2.1). In practice, it happens that δx is of stochastic nature. For example, the priori information on the FE is an zero mean

,

, Xei, <sup>2</sup> ∈ Rn�ð Þ <sup>n</sup>�<sup>k</sup> . Multiplying the right of EVk by Xei,<sup>1</sup> yields Xei, <sup>1</sup>J1, i.e., we

ei, <sup>1</sup>; <sup>X</sup><sup>~</sup> <sup>T</sup> ei, 2 h i

; <sup>j</sup>λ<sup>j</sup> <sup>&</sup>lt; <sup>j</sup>λ<sup>i</sup>�<sup>1</sup><sup>j</sup> � �: (14)

ei ¼ Φ (15)

ei is the matrix inverse of Xei (see Golub and

(16)

$$\mathcal{S}\_s(\delta \mathbf{x}) \coloneqq \left\{ \delta \mathbf{x} \; : \; \| \delta \mathbf{x} \| \_H = 1 \right\} \tag{19}$$

Definition 4.1. The optimal stochastic perturbation (OSP) δxo is the solution of the extremal problem

$$J(\delta \mathbf{x}) = \|\Phi \delta \mathbf{x}\|\_{H} \to \mathbf{max}\_{\delta \mathbf{x}} \tag{20}$$

under the constraint (19). One can prove

Lemma 4.1. For δx∈ Ssð Þ δx , there exists δy ¼ δy1;…; δyn � �<sup>T</sup> <sup>∈</sup> Ssð Þ <sup>δ</sup><sup>x</sup> such that <sup>δ</sup><sup>x</sup> <sup>¼</sup> <sup>P</sup><sup>n</sup> <sup>k</sup>¼<sup>1</sup> vkδyk.

Lemma 4.2. The optimal perturbation in the sense (20) and (19) is

$$
\delta \mathbf{x}^o = \psi \upsilon\_1
$$

where ψ is a RV with zero mean and unit variance, v<sup>1</sup> is the first right SV of Φ.

Comment 4.1. Comparing the ODP with OSP shows that if the ODS is the first right SV (defined up to the sign), the OSP is an ensemble of vectors lying in the subspace of the first right SV with the lengths being the samples of the RV of zero mean and unit variance.

Introduce

$$\Phi\_1 := \Phi - \sigma\_1 u\_1 v\_1^T$$

and consider the objective function

$$J(\delta \mathbf{x}) = \|\Phi\_1 \delta \mathbf{x}\|\_H \to \mathbf{max}\_{\delta \mathbf{x}\prime} \tag{21}$$

Lemma 4.3. The optimal perturbation in the sense (21) and (19) is <sup>δ</sup>xo <sup>¼</sup> <sup>ψ</sup>v2, where <sup>ψ</sup> is an RV with zero mean and unit variance, v<sup>2</sup> is the first right SV of Φ.

By iteration, for

$$\Phi\_{\mathbf{i}} \coloneqq \Phi\_{\mathbf{i}-1} - \sigma\_{\mathbf{i}} u\_{\mathbf{i}} \upsilon\_{\mathbf{i}}^T, \mathbf{i} = 1, \ldots, n - 1; \Phi\_0 = \Phi. \tag{22}$$

E Δ<sup>j</sup>

c is a small positive value, from Eq. (24)

<sup>Δ</sup>Jð Þ¼ <sup>θ</sup><sup>0</sup> <sup>c</sup>Δ<sup>T</sup>

Dividing both sides of the last equality by δθ<sup>k</sup> ¼ cΔ<sup>k</sup> implies

DJð Þ θ<sup>0</sup>

<sup>J</sup>ð Þ <sup>θ</sup><sup>0</sup> <sup>Δ</sup><sup>k</sup> h i <sup>¼</sup> 0 since there exists a finite <sup>D</sup><sup>2</sup>

L i.i.d sample estimates for the gradient vector at θ ¼ θ0,

DJð Þ<sup>l</sup> ð Þ¼ <sup>θ</sup><sup>0</sup>

th component of the l

mated by averaging L sample gradients in Eq. (29),

ΔJð Þ θ<sup>0</sup> =δθ<sup>k</sup> ¼ DJð Þ θ<sup>0</sup>

where D<sup>2</sup>

E Δ<sup>T</sup> D2

where Δð Þ<sup>l</sup>

<sup>k</sup> is the k

� � <sup>¼</sup> <sup>0</sup>, E <sup>Δ</sup><sup>j</sup>

� �<sup>2</sup> <sup>¼</sup> <sup>1</sup>, E <sup>Δ</sup>�<sup>1</sup>

j

Suppose Jð Þ θ is infinitely differentiable at θ ¼ θ0. Using a Taylor series expansion, <sup>Δ</sup><sup>J</sup> <sup>≔</sup> <sup>J</sup> <sup>θ</sup><sup>0</sup> <sup>þ</sup> δθ � � � <sup>J</sup>ð Þ¼ <sup>θ</sup><sup>0</sup> δθTDJð Þþ <sup>θ</sup><sup>0</sup> ð Þ <sup>1</sup>=<sup>2</sup> δθTD<sup>2</sup>

Jð Þ θ<sup>0</sup> is the Hessian matrix computed at θ ≔ θ0. For the choice

DJð Þþ θ<sup>0</sup> c

<sup>T</sup>Δ<sup>k</sup>

Δk

TE <sup>Δ</sup><sup>k</sup> � � <sup>þ</sup> ð Þ <sup>c</sup>=<sup>2</sup> <sup>E</sup> <sup>Δ</sup><sup>T</sup>

One sees that from the assumptions on <sup>Δ</sup>, <sup>E</sup> <sup>Δ</sup><sup>k</sup> � � <sup>¼</sup> ð Þ <sup>0</sup>;…; <sup>1</sup>; …; <sup>0</sup> <sup>T</sup> it follows DJ uð Þ<sup>0</sup>

<sup>∂</sup>Jð Þ <sup>θ</sup><sup>0</sup> <sup>=</sup>∂θk. Moreover, as all the moments of the Bernoulli variables <sup>Δ</sup><sup>i</sup> and <sup>Δ</sup>�<sup>1</sup>

k

The result expressed by Eq. (28) constitutes a basis for approximating the gradient vector by simultaneous perturbation. The left of Eq. (28) can be easily approximated by noticing that for an ensemble of <sup>L</sup> i.i.d samples <sup>Δ</sup>ð Þ<sup>1</sup> ;…; ;Δð Þ <sup>L</sup> h i, we can generate the corresponding ensemble of

> ;…; ΔJ ð Þ<sup>l</sup> ð Þ <sup>θ</sup><sup>0</sup> cΔð Þ<sup>l</sup> n

th sample Δð Þ<sup>l</sup>

" #<sup>T</sup>

<sup>E</sup> <sup>Δ</sup>Jð Þ <sup>θ</sup><sup>0</sup> δθ�<sup>1</sup>

ΔJ ð Þ<sup>l</sup> ð Þ <sup>θ</sup><sup>0</sup> cΔð Þ<sup>l</sup> 1

Taking the mathematical expectation for both sides of the last equation yields

2 =2 � �Δ<sup>T</sup>

≔ Δ1Δ�<sup>1</sup>

<sup>þ</sup> ð Þ <sup>c</sup>=<sup>2</sup> <sup>Δ</sup>k,T

� � <sup>¼</sup> <sup>0</sup>, E <sup>Δ</sup>�<sup>1</sup>

On Optimal and Simultaneous Stochastic Perturbations with Application to Estimation of High-Dimensional…

j � �<sup>2</sup>

δθ <sup>≔</sup> δθ1; …; δθ<sup>n</sup><sup>θ</sup> ð Þ<sup>T</sup> <sup>¼</sup> <sup>c</sup>Δ,<sup>Δ</sup> <sup>¼</sup> <sup>Δ</sup>1;Δ2;…;Δ<sup>n</sup><sup>θ</sup> ð ÞT, (25)

D2

D2

<sup>E</sup> <sup>Δ</sup>Jð Þ <sup>θ</sup><sup>0</sup> δθ�<sup>1</sup>

Jð Þ θ<sup>0</sup> , one concludes that

� � <sup>¼</sup> <sup>∂</sup>Jð Þ <sup>θ</sup><sup>0</sup> <sup>=</sup>∂θ<sup>k</sup> <sup>þ</sup> O c<sup>2</sup> � � (28)

D2

k � � <sup>¼</sup>

<sup>k</sup> ; …; 1;…; ΔnΔ�<sup>1</sup>

Jð Þ θ<sup>0</sup> Δ þ …

Jð Þ θ<sup>0</sup> Δ þ …

k � �<sup>T</sup> (26)

<sup>J</sup>ð Þ <sup>θ</sup><sup>0</sup> <sup>Δ</sup><sup>k</sup> h i <sup>þ</sup> … (27)

, l ¼ 1, 2, …, L (29)

. The left of Eq. (28) is then well approxi-

TE <sup>Δ</sup><sup>k</sup> � � <sup>¼</sup>

<sup>i</sup> are finite,

¼ 1, j ¼ 1, 2, …, n (23)

http://dx.doi.org/10.5772/intechopen.77273

Jð Þ θ<sup>0</sup> δθ þ … (24)

71

applying Lemma 4.3 with slight modifications, one finds that the OSP for Φk, k ¼ 0, 1, …, n � 1 are ψvk, k ¼ 1, 2, …n:, where ψ is an RV with zero mean and unit variance, vk is the k th right SV of Φ.

Theorem 4.1. The optimal perturbation for the matrix Φ<sup>i</sup>�<sup>1</sup>, i ¼ 1, …, n is ψvi, where ψ is an RV with zero mean and unit variance, vk is the kth right SV of Φ.
