**7. General multivariate linear model**

In this section, we consider a general multivariate linear model as follows. Let **Y** be an *n* × *p* observation matrix whose rows are independently distributed as *p*-variate normal distribution with a common covariance matrix Σ. Suppose that the mean of **Y** is given as

$$\mathbf{E}(\mathbf{Y}) = \mathbf{A} \boldsymbol{\Theta} \mathbf{X}^{\prime},\tag{7.1}$$

where **A** is an *n* × *k* given matrix with rank *k*, **X** is a *p* × *q* matrix with rank *q*, and Θ is a *k* × *q* unknown parameter matrix. For a motivation of (7.1), consider the case when a single variable *y* is measured at *p* time points *t*1, …, *tp* (or different conditions) on *n* subjects chosen at random from a group. Suppose that we denote the variable *y* at time point *tj* by *yj* . Let the observations *yi*1, …, *yip* of the *i*th subject be denoted by

$$\mathbf{y}\_i = (\mathbf{y}\_{i1}, \dots, \mathbf{y}\_{ip})', \quad i = 1, \dots, n.$$

If we consider a polynomial regression of degree *q* − 1 of *y* on the time variable *t*, then

$$\operatorname{E}(\mathbf{y}\_i) = \mathbf{X}\boldsymbol{\theta}\_\*$$

where

$$\mathbf{X} = \begin{pmatrix} 1 & t\_1 & \cdots & t\_1^{q-1} \\ \vdots & \vdots & \cdots & \vdots \\ 1 & t\_p & \cdots & t\_p^{q-1} \\ \end{pmatrix}, \quad \mathbf{\theta} = \begin{pmatrix} \theta\_1 \\ \theta\_2 \\ \vdots \\ \theta\_q \\ \end{pmatrix}.$$

If there are *k* different groups and each group has a polynomial regression of degree *q* − 1 of *y*, we have a model given by (7.1). From such motivation, the model (7.1) is also called a growth curve model. For its detail, see [14].

Now, let us consider to derive LRC for a general linear hypothesis

$$H\_{\boldsymbol{\beta}}: \mathbf{C} \boldsymbol{\Theta} \mathbf{D} = \mathbf{O},\tag{7.2}$$

against alternatives *Kg* :**C**Θ**D**≠**O**. Here, **C** is a *c* × *k* given matrix with rank *c*, and **D** is a *q* × *d* given matrix with rank *d*. This problem was discussed by [15–17]. Here, we obtain LRC by reducing it to the problem of obtaining LRC for a general linear hypothesis in a multivariate linear model. In order to relate the model (7.1) to a multivariate linear model, consider the transformation from **Y** to (**U V**):

$$\mathbf{(U\,V)=YG},\quad\mathbf{G}=(\mathbf{G}\_1\,\mathbf{G}\_2),\tag{7.3}$$

where **G**1 = **X**(**X**′**X**) − 1, **<sup>G</sup>**<sup>2</sup> <sup>=</sup>**X**˜ , and **X**˜ are a *<sup>p</sup>* × (*p* − *q*) matrix satisfying **X**˜**′ <sup>X</sup>**=**O** and **X**˜**′ <sup>X</sup>**˜ <sup>=</sup>**<sup>I</sup>** *<sup>p</sup>*−*<sup>q</sup>*. Then, the rows of (**U V**) are independently distributed as *p*-variate normal distributions with means

$$[\operatorname{E}[(\mathbf{U}\,\mathbf{V})] = (\mathbf{A}\Theta\,\mathbf{O})\_\* $$

and the common covariance matrix

$$
\Psi = \mathbf{G}' \Sigma \mathbf{G} = \begin{pmatrix} \mathbf{G}'\_1 \mathbf{\Xi} \mathbf{G}\_1 & \mathbf{G}'\_1 \mathbf{\Xi} \mathbf{G}\_2 \\ \mathbf{G}'\_2 \mathbf{\Xi} \mathbf{G}\_1 & \mathbf{G}\_2 \mathbf{\Xi} \mathbf{G}\_2 \end{pmatrix} = \begin{pmatrix} \mathbf{\Psi}\_{11} & \mathbf{\Psi}\_{12} \\ \mathbf{\Psi}\_{21} & \mathbf{\Psi}\_{22} \end{pmatrix}.
$$

This transformation can be regarded as one from *y* = (*y*1, …, *yp*)′ to a *q*-variate main variable *u* = (*u*1, …, *uq*)′ and a (*p* − *q*)-variate auxiliary variable *v* = (*v*1, …, *vp* <sup>−</sup> *<sup>q</sup>*)′. The model (7.1) is equivalent to the following joint model of two components:

**1.** The conditional distribution of **U** given **V** is

$$\mathbf{U} \mid \mathbf{V}: \quad \mathrm{N}\_{\boldsymbol{w} \times \boldsymbol{q}}(\mathbf{A}^{\ast} \mathbf{E}, \mathbf{\varPsi}\_{1 \sqcup \boldsymbol{2}}). \tag{7.4}$$

**2.** The marginal distribution of **V** is

$$\mathbf{V} \colon \quad \mathrm{N}\_{\mathrm{u}\times(p-q)}(\mathbf{O}, \mathbf{\Psi}\_{22}), \tag{7.5}$$

where

Similarly, **Q**2 is idempotent. Using **P1***<sup>n</sup>*

**7. General multivariate linear model**

*yi*1, …, *yip* of the *i*th subject be denoted by

where

Further, under *H*2 ⋅ 1,

158 Applied Linear Algebra in Action

**P**<sup>A</sup> =**P**A**P1***<sup>n</sup>*

2

=**P1***<sup>n</sup>*

21 21 121 ( ) = , ( )= . **Q Q Q Q QQ Q** - - ×- **O**

E( | ) ( )E( | ) = . 21 2 1 21 **XX Q Q XX** ¢ - **O**

In this section, we consider a general multivariate linear model as follows. Let **Y** be an *n* × *p* observation matrix whose rows are independently distributed as *p*-variate normal distribution

where **A** is an *n* × *k* given matrix with rank *k*, **X** is a *p* × *q* matrix with rank *q*, and Θ is a *k* × *q* unknown parameter matrix. For a motivation of (7.1), consider the case when a single variable *y* is measured at *p* time points *t*1, …, *tp* (or different conditions) on *n* subjects chosen at random

*i i ip* = ( , , ) , = 1, , . <sup>1</sup> *y yyi n* K K ¢

E( ) = , *y<sup>i</sup>* **X**q

1 1

q

1

 æ ö æ ö ç ÷ ç ÷ ç ÷ ç ÷ ç ÷ ç ÷ ç ÷ è ø ç ÷

= ,= .

*q p p*

MML M <sup>M</sup> <sup>L</sup>

*q*


2

q

q

*q*

è ø

q

If we consider a polynomial regression of degree *q* − 1 of *y* on the time variable *t*, then

1 1

L

*t t*

*t t*

1

**X**

1

with a common covariance matrix Σ. Suppose that the mean of **Y** is given as

from a group. Suppose that we denote the variable *y* at time point *tj*

, we have **Q**1**Q**<sup>2</sup> =**Q**2**Q**<sup>1</sup> =**Q**1 , and hence

E( ) = , **Y X A**Q ¢ (7.1)

by *yj*

. Let the observations

$$\begin{aligned} \mathbf{A}^\* &= (\mathbf{A} \,\mathbf{V}), \quad \mathbf{E} = \begin{pmatrix} \mathbf{O} \\ \mathbf{T} \end{pmatrix}, \\\ \mathbf{T} &= \mathbf{\Psi} \mathbf{\!\!\!\/ }\_{22}^{-1} \mathbf{\!\!\!\/ }\_{21} \mathbf{\!\!\!\/ }\_{11} = \mathbf{\!\!\!\/ }\_{11} - \mathbf{\!\!\!\/ }\_{12} \mathbf{\!\!\/ }\_{22} \mathbf{\!\!\/ }\_{21} \mathbf{\!\!\/ }\_{21}. \end{aligned}$$

Before we obtain LRC, first we consider the MLEs in (7.1). Applying a general theory of multivariate linear model to (7.4) and (7.5), the MLEs of Ξ , Ψ11⋅2 , and Ψ22 are given by

$$
\hat{\Xi} = (\mathbf{A}^\* \mathbf{A}^\*)^{-1} \mathbf{A}^\* \mathbf{U}, \quad n \mathbf{\varPsi}\_{11:2} = \mathbf{U} (\mathbf{I}\_n - \mathbf{P}\_{\Lambda^\*}) \mathbf{U}, \quad n \mathbf{\varPsi}\_{22} = \mathbf{V} \mathbf{V}. \tag{7.6}
$$

Let

$$\mathbf{S} = \mathbf{Y}'(\mathbf{I}\_n - \mathbf{P}\_\lambda)\mathbf{Y}, \quad \mathbf{W} = \mathbf{G}'\mathbf{S}\mathbf{G} = (\mathbf{U}\,\mathbf{V})'(\mathbf{I}\_n - \mathbf{P}\_\lambda)(\mathbf{U}\,\mathbf{V}),$$

and partition **W** as

$$\mathbf{W} = \begin{pmatrix} \mathbf{W}\_{11} & \mathbf{W}\_{12} \\ \mathbf{W}\_{21} & \mathbf{W}\_{22} \end{pmatrix}, \quad \mathbf{W}\_{12} : q \times (p - q).$$

**Theorem 7.1***For ann* × *pobservation matrix***Y***, assume a general multivariate linear model given by* (7.1)*. Then:*

1. *The MLE* **Θ** ^ *of* **<sup>Θ</sup>** *is given by*

$$
\hat{\boldsymbol{\Theta}} = \mathbf{A} (\mathbf{A}' \mathbf{A})^{-1} \mathbf{A}' \mathbf{Y} \mathbf{S}^{-1} \mathbf{X} (\mathbf{X} \mathbf{S}^{-1} \mathbf{X})^{-1},
$$

2. *The MLE* **Ψ** ^ <sup>11</sup>⋅<sup>2</sup> *of* **Ψ**11⋅<sup>2</sup> *is given by*

$$m\mathbf{\hat{P}}\_{11\cdot 2} = \mathbf{W}\_{11\cdot 2} = (\mathbf{X'}\mathbf{S}^{-1}\mathbf{X})^{-1}.$$

*Proof.* The MLE of **Ξ** is Ξ ^ =(**A**\* ′ **A**\*)−<sup>1</sup> **A**\* ′ **U**. The inverse formula (see (5.5)) gives

$$\begin{split} \mathbf{O} &= (\mathbf{A}^\* \mathbf{A}^\*)^{-1} = \begin{pmatrix} (\mathbf{A}^\prime \mathbf{A})^{-1} & \mathbf{O} \\ \mathbf{O} & \mathbf{O} \end{pmatrix} + \begin{pmatrix} -(\mathbf{A}^\prime \mathbf{A})^{-1} \mathbf{A}^\* \mathbf{V} \\ \mathbf{I}\_{p-q} \end{pmatrix} [\mathbf{V}^\prime (\mathbf{I}\_s - \mathbf{P}\_\lambda) \mathbf{V}]^{-1} \begin{pmatrix} -(\mathbf{A}^\prime \mathbf{A})^{-1} \mathbf{A}^\* \mathbf{V} \\ \mathbf{I}\_{p-q} \end{pmatrix}, \\ &= \begin{pmatrix} \mathbf{O}\_{11} & \mathbf{O}\_{12} \\ \mathbf{O}\_{21} & \mathbf{O}\_{22} \end{pmatrix}. \end{split}$$

Therefore, we have

$$\begin{split} \hat{\boldsymbol{\Theta}} &= (\mathbf{Q}\_{11} \mathbf{A}^{\prime} + \mathbf{Q}\_{12} \mathbf{V}^{\prime}) \mathbf{U} \\ &= (\mathbf{A}^{\prime} \mathbf{A})^{-1} \mathbf{A}^{\prime} \mathbf{Y} \mathbf{G}\_{1} - (\mathbf{A}^{\prime} \mathbf{A})^{-1} \mathbf{A}^{\prime} \mathbf{Y} \mathbf{G}\_{2} (\mathbf{G}\_{2}^{\prime} \mathbf{S} \mathbf{G}\_{2})^{-1} \mathbf{G}\_{2}^{\prime} S \mathbf{G}\_{1} .\end{split}$$

Using

\*

\* \*1\*

Let

and partition **W** as

160 Applied Linear Algebra in Action

(7.1)*. Then:*

1. *The MLE* **Θ**

2. *The MLE* **Ψ**

^

*Proof.* The MLE of **Ξ** is Ξ

^ *of* **<sup>Θ</sup>** *is given by*

<sup>11</sup>⋅<sup>2</sup> *of* **Ψ**11⋅<sup>2</sup> *is given by*

^ =(**A**\* ′ **A**\*)−<sup>1</sup> **A**\* ′

11 12 21 22

= .

**Q Q Q Q**

æ ö ç ÷ è ø **A**

**A V**

1 1 22 21 11 2 11 12 22 21

æ ö ç ÷ è ø = -

G G Y Y Y Y Y Y Y

Q

= , . - - ×

Before we obtain LRC, first we consider the MLEs in (7.1). Applying a general theory of multivariate linear model to (7.4) and (7.5), the MLEs of Ξ , Ψ11⋅2 , and Ψ22 are given by

= ( ) , = = ( ) ( )( ), *n n* ¢ ¢¢ - - **A A S Y Y W S UV UV I P GG I P**

12

= , : ( ). *q pq*

ç ÷ ´ -

**Theorem 7.1***For ann* × *pobservation matrix***Y***, assume a general multivariate linear model given by*

1 1 11 ˆ = ( ) ( ). - - -- Q **A AA A X X X** ¢¢ ¢ **YS S**

1 1 *n* 11 2 11 2 = =( ) . - - × × Y **W S X X** ¢

\* \*1 1

**Q AA V V**

¢ - -

( ) ( ) ( ) =( ) = [ ( )]

**<sup>O</sup> I P O O I I A A AA AV AA AV**

**U**. The inverse formula (see (5.5)) gives

1 1 1


æ ö ¢ æö æö - - ¢ ¢ ¢ ¢ + - ç÷ ç÷ ¢ ç ÷ è ø èø èø

A


*n p q p q*

<sup>×</sup> X - ¢ ¢ *<sup>n</sup> n n* **<sup>A</sup> A A A U U U VV** <sup>Y</sup> **I P** <sup>Y</sup> (7.6)

= ( ), = ,

X

11 2 \* <sup>22</sup> ˆ =( ) , = ( ) , = . ¢ ¢ -

11 12

æ ö

è ø **W W W W W W**

21 22

$$\mathbf{G}\_2(\mathbf{G}\_2'\mathbf{S}\mathbf{G}\_2)^{-1}\mathbf{G}\_2' = \mathbf{S}^{-1} - \mathbf{G}\_1(\mathbf{G}\_1'\mathbf{S}^{-1}\mathbf{G}\_2)^{-1}\mathbf{G}\_1'\mathbf{S}^{-1},$$

we obtain 1. For a derivation of 2, let **B**=(**I***<sup>n</sup>* −**PA**)**V**. Then, using **P <sup>A</sup>\***=**PA** <sup>+</sup> **<sup>P</sup><sup>B</sup>** , the first expression of (1) is obtained. Similarly, the second expression of (2) is obtained.

**Theorem 7.2***Letλ* = *Λ<sup>n</sup>*/2*be the LRC for testing the hypothesis*(7.2) *in the generalized multivariate linear model* (7.1)*. Then,*

$$\Lambda = |\mathbf{S}\_e| / \langle |\mathbf{S}\_e + \mathbf{S}\_h|, \rangle$$

*where*

$$\mathbf{S}\_s = \mathbf{D}'(\mathbf{X}\mathbf{S}^{-1}\mathbf{X})^{-1}\mathbf{D}, \quad \mathbf{S}\_h = (\mathbf{C}\hat{\Theta}\mathbf{D})(\mathbf{C}\mathbf{R}\mathbf{C}')^{-1}\mathbf{C}\hat{\Theta}\mathbf{D}$$

*and*

$$\begin{split} \mathbf{R} &= (\mathbf{A'A})^{-1} + (\mathbf{A'A})^{-1} \mathbf{A'} \mathbf{Y} \mathbf{S}^{-1} \{ \mathbf{S} - \mathbf{X'} (\mathbf{X} \mathbf{S}^{-1} \mathbf{X'})^{-1} \mathbf{X} \}, \\ &\times \mathbf{S}^{-1} \mathbf{Y'} \mathbf{A} (\mathbf{A'A})^{-1} .\end{split}$$

*Here***Θ** ^ *is given in Theorem* 7.1.1. *Further, the null distribution is <sup>Λ</sup>d*(*c*, *n* − *k* − (*p* − *q*))*.*

*Proof.* The test of *Hg* in (7.2) against alternatives *Kg* is equivalent to testing

$$H\_s: \mathbf{C}' \mathbf{E} \mathbf{D} = \mathbf{O} \tag{7.7}$$

under the conditional model (7.4), where **C**\* = (**C O**). Since the distribution of **V** does not depend on *Hg*, the LR test under the conditional model is the LR test under the unconditional model. Using a general result for a general linear hypothesis given in Theorem 4.1, we obtain

$$\Lambda = |\tilde{\mathbf{S}}\_{\boldsymbol{e}}| \mid \boldsymbol{\wedge} | \tilde{\mathbf{S}}\_{\boldsymbol{e}} + \tilde{\mathbf{S}}\_{h} \mid, \boldsymbol{\wedge}$$

where

$$\begin{aligned} \tilde{\mathbf{S}}\_{\circ} &= \mathbf{D}^{\prime} \mathbf{U} (\mathbf{I}\_{\circ} - \mathbf{A}^{\prime} (\mathbf{A}^{\prime} \mathbf{A}^{\ast})^{-1} \mathbf{A}^{\ast}) \mathbf{U} \mathbf{D}, \\ \tilde{\mathbf{S}}\_{\circ} &= (\mathbf{C} \hat{\Xi} \mathbf{D}) (\mathbf{C}^{\prime} (\mathbf{A}^{\prime} \mathbf{A}^{\ast})^{-1} \mathbf{C}^{\ast})^{-1} \mathbf{C} \hat{\Xi} \mathbf{D}. \end{aligned}$$

By reduction similar to those of MLEs, it is seen that **S**˜*<sup>e</sup>* <sup>=</sup>**S***<sup>e</sup>* and **S**˜ *<sup>h</sup>* <sup>=</sup>**S***<sup>h</sup>* . This completes the proof.
