**2. System description**

Consider the following SISO linear system expressed by Auto-Regressive and Moving Average Model (ARMA) (Astrom & Wittenmark, 1994):

$$\begin{aligned} y(k) &+ a\_1(k)y(k-1) + \dots + a\_n(k)y(k-n) \\ &= b\_1(k)u(k-1) + \dots + b\_m(k)u(k-m) + c(k) \end{aligned} \tag{1}$$

where *u k*( ) and *y*( ) *k* are input and output of the system respectively, stochastic noise sequence { ( )} *e k* is Gaussian and white with zero-mean and covariance *R*, and uncorrelated with *y*( ) *k* , *y*( - 1) *k* and *u k*( ) , *u k*( - 1) . Let

Identifiability of Quantized Linear Systems 259

$$\Theta(k) = [b\_1(k)\cdots b\_m(k)\, a\_1(k)\cdots a\_n(k)]^\text{T} \tag{2}$$

be the parameter vector, where <sup>T</sup> ( ) denotes the operation of transposition, and let

$$F(k) = \left[\mu(k-1)\cdots\mu(k-m) - y(k-1)\cdots - y(k-m)\right]^1\tag{3}$$

then system (1) can be described as

258 Stochastic Modeling and Control

systems.

is analyzed.

**2. System description** 

information criterion, if the mutual information between actual output and model output is greater than zero. However, the concept of identifiability in (Durgaryan & Pashchenko, 2001) is defined in principle, based on which there is no practical result. Reference (Zhang, 2003; Zhang & Sun, 1996; Baram & Kailath, 1988) have discussed the problem of states estimability, which is related closely with parameter identifiability, for that inputoutput description of linear systems with Gauss-Markov parameters can be transformed to state space model, and then the problem of parameter identifiability can be treated as state estimability. Reference (Zhang, 2003) has proposed the definition of parameter identifiability under the criterion of minimum maximum error entropy (MMSE) estimation referring to the definition of states estimability, and also obtained some useful conclusions. Reference (Wang & Zhang, 2011) has studied the parameter identifiability of linear systems under access constraints. However, there is few work on that for quantized

This paper mainly analyzes the parameter identifiability of quantized linear systems with Gauss-Markov parameters from information theoretic point of view. The definition of parameter identifiability proposed in (Zhang, 2003) is reviewed: the linear system with Gauss-Markov parameters is identifiable, if and only if the mutual information between the actual value and estimates of parameters is greater than zero, which is extended to quantized systems by considering the intrinsic property of the system. Then the parameter identifiability of linear systems with quantized outputs is analyzed and the criterion of parameter identifiability is obtained based on the measure of mutual information. Furthermore, the convergence property of the quantized parameter identifiability Gramian

The rest of the paper is organized as follows: In section 2, we introduce the model that we are interested in; Section 3 discusses the existing definition of parameter identifiability, proposes our new definition, and gives analytic conclusion focusing on quantized linear systems with Gauss-Markov parameters; The convergence property of Gramian matrix of parameter identifiability for quantized systems is discussed in section 4; Section 5 and

Consider the following SISO linear system expressed by Auto-Regressive and Moving

 *b kuk b kuk m ek*

 

where *u k*( ) and *y*( ) *k* are input and output of the system respectively, stochastic noise sequence { ( )} *e k* is Gaussian and white with zero-mean and covariance *R*, and uncorrelated

( ) ( - 1) ( ) ( - ) ( ) *n m*

(1)

( ) ( ) ( - 1) ( ) ( - )

*yk a kyk a kyk n*

6 are illustrative simulation and conclusion, respectively.

Average Model (ARMA) (Astrom & Wittenmark, 1994):

with *y*( ) *k* , *y*( - 1) *k* and *u k*( ) , *u k*( - 1) . Let

1 1

$$y(k) = F^{\top}(k)\Theta(k) + e(k)\tag{4}$$

Suppose that the parameter*θ*( ) *k* can be modeled by a Gauss-Markov process, i.e.,

$$
\Theta(k+1) = A\Theta(k) + Bw(k) \tag{5}
$$

where *A*, *B* are known matrices with appropriate dimensions; noise sequence { ( )} *w k* is Gaussian and white with zero-mean and covariance *Q*; initial value of the parameter *θ*(0) is Gaussian with mean *θ* and covariance *Π*(0) . Suppose that *e k*( ) , *w k*( ) and *θ*(0) are mutually uncorrelated. Hence, linear system (1) with Gauss-Markov parameters can be described by (4) and (5), i.e.,

$$\begin{cases} \mathcal{O}(k+1) = A\mathcal{O}(k) + Bw(k) \\ y(k) = F^\top(k)\mathcal{O}(k) + \varepsilon(k) \end{cases} \tag{6}$$

Due to bandwidth constraint of the channel, quantization is required. The discussion in the present paper does not focus on a special quantizer, but on general *N*-level quantization (Curry 1970; Gray and Neuhoff 1998) which can be described as:

$$\mathcal{Y}\_q(k) = \mathcal{Q}(y(k)) = z\_{l'}, \text{ for } y(k) \in \delta\_{l'}, l = 1, 2, \cdots, N \tag{7}$$

where *Q*( ) is the general quantizer, q 12 () { , , , } *<sup>N</sup> y k zz z* denote the quantizer outputs with , = 1, , *<sup>l</sup> zl N* as the reproduction values; = ( , ], = 1,2, , *l l l+1 δ aa l N* denote the quantization intervals, where +1 =1 *N <sup>i</sup> <sup>i</sup> <sup>a</sup>* , 1 2 +1 - = < < < =+ *<sup>N</sup> aa a* are the thresholds of the quantizer.

The channel is assumed to be lossless. q 12 () { , , , } *<sup>N</sup> y k zz z* is transmitted and then received at the channel receiver. , = 1,2, , *<sup>i</sup> zi N* are symbols denoting the *i*th quantization interval and not necessarily real numbers, hence, further decoding is required, as follows

$$y\_q^\*(k) = D\_k(y\_q(k))\tag{8}$$

where ( ) *Dk* is assumed to be a one to one mapping. A common decoding method (Curry, 1970) is

$$D\_k(y\_q(k)) = \operatorname{E}\{y(k) \mid y\_q(k) = z\_l\}, \ l = 1, 2, \dots, N$$

where E{.} is the operation of expectation.

## **3. Parameter identifiability**

#### **3.1. Definition of parameter identifiability**

Reference (Zhang, 2003) proposed the definition of parameter identifiability for system (6) referring to the definition of state estimability under MMEE.

Let ˆ *θ*( ) *k* be the MMEE estimation of *θ*( ) *k* based on *F k*( ) , and ˆ *θ*( )= ( ) - ( ) *k θ k θ k* be the estimation error. Define the prior and posterior mean-square estimation error matrices respectively as

$$\begin{aligned} \Pi(k) &= \mathbb{E}\{ (\boldsymbol{\varTheta}(k) \cdot \overline{\boldsymbol{\varTheta}}(k)) (\boldsymbol{\varTheta}(k) \cdot \overline{\boldsymbol{\varTheta}}(k))^\top \} \\\\ P(k) &= \mathbb{E}\{ \tilde{\boldsymbol{\varTheta}}(k) \tilde{\boldsymbol{\varTheta}}^\top(k) \} \end{aligned}$$

where*θ*( ) *k* is the mean of *θ*( ) *k* , i.e. *θ*( )=E{ ( )} *k θ k* .

**Definition 1**(Zhang, 2003): The linear system (1) with Gauss-Markov parameters (i.e. system (6)) is identifiable, if and only if

$$I(\Theta(k); \hat{\Theta}(k)) \ge 0, \forall k \ge n+m-1 \tag{9}$$

Identifiability of Quantized Linear Systems 261

ˆ *H*( ( )) > ( ( )), + - 1 *θ k H θ k k nm* (12)

( ( ); ) > 0, k + - 1 *<sup>k</sup> I θ kY n m* (13)

where *H*( ) denotes entropy, i.e. the prior error entropy *H*( ( )) *θ k* is strictly greater than

2. Definition 1 considers the mutual information between the actual value and estimate of parameters, so it relies on the estimation principle, while Definition 2 takes into account the intrinsic property of the system independent of the estimator used. If Definition 2 is adopted to analyze unquantized system (6), the identifiability condition (11) turns into

where <sup>T</sup> = [ (0) (1) ( )] *<sup>k</sup> Y y y yk* , and condition (9) is equivalent to (13) for linear Gaussian system (6) (Zhang, 2003). Hence, in some sense, Definition 2 is a more general one than

Mutual information *I*(;) (Cover & Thomas, 2006) is a measure of the information amount commonly contained in, and the statistic dependence between two random variables. *I*(;) 0 with equality if and only if these two random variables are independent. Therefore (11) based on the information theoretic Definition 2 indicates that system is parameter identifiable if and only if any direction of parameter space is not orthogonal to all the past

q q *y* ( ) E{ ( )} *j yj* .


> \* \* q q \* q

*k j k j kji*


*A θ j θ j A Bw j i y j*

=0

*i*

= E{[ ( ( ) - ( )) + ( + )] ( )}

q


= E{( ( ) - ( )) ( )}

*A θ j θ j yj*

E{( ( ) - ( ))( ( ) - ( ))}

*θ k θ k yj yj*

q


= E{ '( ) ( )}

*A θ jyj*

= E{( ( ) - ( )) ( )}

*θ k θ k yj*

q q *<sup>g</sup>* E{( ( ) - ( ))( ( ) - ( ))} 0, , + - 1 *<sup>θ</sup> <sup>k</sup> <sup>θ</sup> k yj yj j k knm* (14)

*<sup>θ</sup> k A <sup>θ</sup> j A Bw j i* **<sup>j</sup>** (15)


q

(17)

**3.2. Identifiability analysis of quantized linear systems** 

(quantized) measurements (Baram & Kailath, 1988), i.e. <sup>+</sup> , 0, *n m g g* **R**

<sup>q</sup> *<sup>y</sup>* ( )*<sup>j</sup>* , i.e. \* \*

T \* \*

From the Gauss-Markov property of the parameters, we have

posterior error entropy ˆ *H*( ( )) *θ k* ;

Definition 1.

where \*

where


=0

*i*

<sup>q</sup> *<sup>y</sup>* ( )*j* is the mean of \*

( + )=0 *<sup>i</sup>*

*A Bw k i* when *j k,* = then

*k j k j*

where *I*(;) denotes mutual information.

Based on Definition 1, the following conclusion was obtained in (Zhang, 2003).

**Lemma 1**(Zhang, 2003): The linear system (1) with Gauss-Markov parameters (i.e. system (6)) is identifiable, if and only if, the identifiability Gramian

$$\mathcal{W}\_k^{\rm id} = \sum\_{j=k}^0 A^{k,j} \Pi(j) F(j) F^\Gamma(j) \Pi(j) (A^{k,j})^\Gamma, \ \forall k \ge n+m-1 \tag{10}$$

has full rank, i.e. id ( )= + *<sup>k</sup> rank W n m* , + - 1 *k nm* .

In the present paper, we propose an alternative definition of parameter identifiability for the quantized system (6)(7)(8) from information theoretic point of view, as follows.

**Definition 2**: The quantized linear system with Gauss-Markov parameters (6)(7)(8) is identifiable, if and only if

$$I(\boldsymbol{\Theta}(k); \boldsymbol{Y}\_{\boldsymbol{q}}^{\*k}) \ge 0, \; \forall \mathbf{k} \ge \mathbf{n} + m - 1 \tag{11}$$

where \* \* \* \*T q qq q = [ (0) (1) ( )] . *<sup>k</sup> Y y y yk*

#### **Remark 1**:

1. From information theory (Cover & Thomas, 2006), condition (11) is equivalent to that

$$H(\Theta(k)) \succeq H(\hat{\Theta}(k)), \forall k \ge n+m-1\tag{12}$$

where *H*( ) denotes entropy, i.e. the prior error entropy *H*( ( )) *θ k* is strictly greater than posterior error entropy ˆ *H*( ( )) *θ k* ;

2. Definition 1 considers the mutual information between the actual value and estimate of parameters, so it relies on the estimation principle, while Definition 2 takes into account the intrinsic property of the system independent of the estimator used. If Definition 2 is adopted to analyze unquantized system (6), the identifiability condition (11) turns into

$$I(\boldsymbol{\Theta}(k); Y^k) \ge 0, \forall \mathbf{k} \ge \boldsymbol{\mu} + \boldsymbol{m} \cdot \mathbf{1} \tag{13}$$

where <sup>T</sup> = [ (0) (1) ( )] *<sup>k</sup> Y y y yk* , and condition (9) is equivalent to (13) for linear Gaussian system (6) (Zhang, 2003). Hence, in some sense, Definition 2 is a more general one than Definition 1.

#### **3.2. Identifiability analysis of quantized linear systems**

260 Stochastic Modeling and Control

Let ˆ

respectively as

**3. Parameter identifiability** 

**3.1. Definition of parameter identifiability** 

where*θ*( ) *k* is the mean of *θ*( ) *k* , i.e. *θ*( )=E{ ( )} *k θ k* .

(6)) is identifiable, if and only if

identifiable, if and only if

**Remark 1**:

where \* \* \* \*T q qq q = [ (0) (1) ( )] . *<sup>k</sup> Y y y yk*

where *I*(;) denotes mutual information.

referring to the definition of state estimability under MMEE.

Reference (Zhang, 2003) proposed the definition of parameter identifiability for system (6)

<sup>T</sup> *Π*( ) = E{( ( ) - ( ))( ( ) - ( )) } *k θ k θ k θ k θ k*

<sup>T</sup> *P k*( ) = E{ ( ) ( )} *θ k θ k*

**Definition 1**(Zhang, 2003): The linear system (1) with Gauss-Markov parameters (i.e. system

**Lemma 1**(Zhang, 2003): The linear system (1) with Gauss-Markov parameters (i.e. system

In the present paper, we propose an alternative definition of parameter identifiability for the

**Definition 2**: The quantized linear system with Gauss-Markov parameters (6)(7)(8) is

1. From information theory (Cover & Thomas, 2006), condition (11) is equivalent to that

Based on Definition 1, the following conclusion was obtained in (Zhang, 2003).

id - - T T

quantized system (6)(7)(8) from information theoretic point of view, as follows.

\*

= ( ) ( ) ( ) ( )( ) , *k j k j*

(6)) is identifiable, if and only if, the identifiability Gramian

0

=

*j k*

*k*

has full rank, i.e. id ( )= + *<sup>k</sup> rank W n m* , + - 1 *k nm* .

ˆ *I*( ( ); ( )) > 0, + - 1 *θ k θ k k nm* (9)

<sup>q</sup> ( ( ); ) > 0, k + - 1 *<sup>k</sup> <sup>I</sup> <sup>θ</sup> kY n m* (11)

*W A <sup>Π</sup> jFjF j <sup>Π</sup> j A k nm* + -1 (10)

*θ*( ) *k* be the MMEE estimation of *θ*( ) *k* based on *F k*( ) , and ˆ *θ*( )= ( ) - ( ) *k θ k θ k* be the estimation error. Define the prior and posterior mean-square estimation error matrices

> Mutual information *I*(;) (Cover & Thomas, 2006) is a measure of the information amount commonly contained in, and the statistic dependence between two random variables. *I*(;) 0 with equality if and only if these two random variables are independent. Therefore (11) based on the information theoretic Definition 2 indicates that system is parameter identifiable if and only if any direction of parameter space is not orthogonal to all the past (quantized) measurements (Baram & Kailath, 1988), i.e. <sup>+</sup> , 0, *n m g g* **R**

$$\log^{\mathsf{T}}\mathrm{E}\{ (\boldsymbol{\Theta}(\boldsymbol{k})\cdot\boldsymbol{\overline{\Theta}}(\boldsymbol{k}))(\boldsymbol{y}\_{\mathrm{q}}^{\ast}(\boldsymbol{j})\cdot\boldsymbol{\overline{y}\_{\mathrm{q}}^{\ast}(\boldsymbol{j})})\}\neq \boldsymbol{0},\,\exists \boldsymbol{j}\leq \boldsymbol{k},\,\forall \boldsymbol{k}\geq \boldsymbol{m}+\boldsymbol{m}\cdot\boldsymbol{1}\tag{14}$$

where \* <sup>q</sup> *<sup>y</sup>* ( )*j* is the mean of \* <sup>q</sup> *<sup>y</sup>* ( )*<sup>j</sup>* , i.e. \* \* q q *y* ( ) E{ ( )} *j yj* .

From the Gauss-Markov property of the parameters, we have

$$\Theta(k) = A^{k \cdot j} \Theta(j) + \sum\_{i=0}^{k \cdot j - 1} A^{k \cdot j \cdot i \cdot i \cdot 1} Bw(j + i) \tag{15}$$

$$
\overline{\Theta}(k) = A^{k-j} \overline{\Theta}(j) \tag{16}
$$

where -1 - -1 =0 ( + )=0 *<sup>i</sup> i A Bw k i* when *j k,* = then

$$\begin{aligned} &\mathbb{E}\{ (\boldsymbol{\Theta}(k) \cdot \overline{\boldsymbol{\Theta}}(k)) (\boldsymbol{y}\_{\neq}^{\*}(j) \cdot \overline{\boldsymbol{y}}\_{\neq}^{\*}(j)) \} \\ &= \mathbb{E}\{ (\boldsymbol{\Theta}(k) \cdot \overline{\boldsymbol{\Theta}}(k)) \cdot \boldsymbol{y}\_{\neq}^{\*}(j) \} \\ &= \mathbb{E}\{ [\boldsymbol{A}^{k \cdot j}(\boldsymbol{\Theta}(j) \cdot \overline{\boldsymbol{\Theta}}(j)) + \sum\_{i=0}^{k \cdot j - 1} \boldsymbol{A}^{k \cdot j - i - 1} \boldsymbol{B} \boldsymbol{w}(j + i)] \cdot \boldsymbol{y}\_{\neq}^{\*}(j) \} \\ &= \boldsymbol{A}^{k \cdot j} \mathbb{E}\{ (\boldsymbol{\Theta}(j) \cdot \overline{\boldsymbol{\Theta}}(j)) \cdot \boldsymbol{y}\_{\neq}^{\*}(j) \} \\ &= \boldsymbol{A}^{k \cdot j} \mathbb{E}\{ \boldsymbol{\theta}^{\*}(j) \cdot \boldsymbol{y}\_{\neq}^{\*}(j) \} \end{aligned} \tag{17}$$

#### 262 Stochastic Modeling and Control

where *θ*'( ) = ( ) - ( ) *j θ j θ j* . Let ' '( ) *θ θ j* , \* \* q q *y y j* ( ) , and the time variable *j* of other relevant time-variant variables (e.g. *F*(*j*), *Π*( )*j* ) is omitted for notational simplicity, then

$$\begin{split} \mathbb{E}\{\boldsymbol{\mathcal{O}}'(\boldsymbol{j})\boldsymbol{y}\_{\neq}^{\star}(\boldsymbol{j})\} \triangleq & \mathbb{E}\{\boldsymbol{\mathcal{O}}' \cdot \boldsymbol{y}\_{\neq}^{\star}\} = \sum\_{l=1}^{N} \int\_{\boldsymbol{\mathcal{O}}' \in \mathbf{R}^{\star \star m}} \boldsymbol{\mathcal{O}} \cdot \boldsymbol{D}\_{j}(\boldsymbol{z}\_{l}) \cdot p(\boldsymbol{\mathcal{O}}', \boldsymbol{y}\_{\neq}^{\star} = \boldsymbol{D}\_{j}(\boldsymbol{z}\_{l})) \mathrm{d}\boldsymbol{\mathcal{O}}^{\star} \\ &= \sum\_{l=1}^{N} \boldsymbol{D}\_{j}(\boldsymbol{z}\_{l}) \int\_{\boldsymbol{\mathcal{O}}' \in \mathbf{R}^{\star \star m}} \boldsymbol{\mathcal{O}} \cdot p(\boldsymbol{\mathcal{O}}', \boldsymbol{y}\_{\neq}^{\star} = \boldsymbol{D}\_{j}(\boldsymbol{z}\_{l})) \mathrm{d}\boldsymbol{\mathcal{O}}^{\star} \end{split} \tag{18}$$

Identifiability of Quantized Linear Systems 263

(23)

(24)

(27)

q q *g* E{( ( ) - ( ))( ( ) - ( ))} 0 *θ k θ k yj yj* is

+1

, + - 1 . *j k k nm*

Combining (22) with (17) and (14), we get that T \* \*

T +1 -

*<sup>N</sup> j l l l k j*

*D z aFj <sup>θ</sup> j a Fj <sup>θ</sup> <sup>j</sup> g A <sup>Π</sup> jFj F j Π jFj R F j Π jFj R F j Π jFj R*

( ) ( ) ( )+ ( ) ( ) ( )+ ( ) ( ) ( )+

TT T =1

idq 2 TT - -

*<sup>N</sup> j l l l*

*D z aFj <sup>θ</sup> j a Fj <sup>θ</sup> <sup>j</sup> <sup>ψ</sup> <sup>j</sup>*

( ) - () () - () () ( ) ( ( )- ( ))

TT T =1

0

=

*j k*

has full rank. We conclude the above analysis as follows.

*k*

parameter identifiable, if and only if

the quantizer, i.e.,

*l*

T T

 

( ( ) ( )+ ( ( ) ( )+ ( ( ) ( )+

*F j)Π j F j R F j)Π j F j R F j)Π jFj R*

= ( ) ( ) ( ) ( ) ( )( ) , + - 1 *k j k j*

**Theorem 1**: The quantized linear system with Gauss-Markov parameters (6)(7)(8) is

1. In Theorem 1, *ψ*( ), = 0,1,2, , *jj k* is defined by the quantizer, while - - T T ( ) ( ) ( ) ( )( ) *k j k j A Π jFjF j Π j A* , *j k* = 0,1,2, , , which is the same part as in the unquantized system, reflects the intrinsic properties of the system. Hence, the full rank requirement of idq *Wk* shows that the parameter identifiability of the quantized system is

2. When quantization level = 1 *<sup>N</sup>* , i.e. 1*<sup>a</sup>* = - , 2 *<sup>a</sup>* = + , *ψ*() 0 *<sup>j</sup>* , then idq <sup>0</sup> *Wk* , condition (26) is not satisfied and the system is not identifiable. This is consistent with the intuition. From (10) and (25), it can be observed that the difference between unquantized estimability Gramian id *Wk* and quantized estimability Gramian idq *Wk* is that the later includes additional weights <sup>2</sup> *ψ* ( ), = 0,1,2, , *jj k* . As a result, it can be seen that besides the situation of quantization level = 1 *<sup>N</sup>* , the matrix idq *Wk* may become

3. The quantizer in Theorem 1 is time-invariant. However, by using the above analysis method, a conclusion similar to Theorem 1 can be derived for time-variant quantizer, except that the weights in the estimability Gramian reflects the time-variant property of

( ( )) ( )- ( ) ( ) ( )- ( ) ( ) ( ) ( ( )- ( ))

*Dzj aj F j <sup>θ</sup> j a jFj <sup>θ</sup> <sup>j</sup> <sup>ψ</sup> <sup>j</sup>*

( ( ) ( )+ ( ( ) ( )+ ( ( ) ( )+

*F j)Π j F j R F j)Π j F j R F j)Π jFj R*

TT T =1

*<sup>N</sup> j l l l*

defined by quantizer and intrinsic properties of the system jointly;

singular due to the property of <sup>2</sup> *<sup>ψ</sup>* ( )*<sup>j</sup>* , though id *Wk* has full rank;

*<sup>W</sup> <sup>ψ</sup> j A <sup>Π</sup> jFjF j <sup>Π</sup> jA k nm* (25)

( ) - () () - () () ( ( ) - ( )) ( ) ( ) 0,

T T

 

idq = + , + - 1 *<sup>k</sup> rank W n m k n m* (26)

T T

  +1

equivalent to

*l*

*l*

Then (23) is equivalent to that

**Remark 2**:

Let

Based on the Bayesian law,

$$\begin{aligned} &p(\mathcal{O}', \mathcal{Y}\_{\mathbf{q}}^{\*} = D\_{j}(z\_{l})) \\ &= p(\mathcal{y}\_{\mathbf{q}}^{\*} = D\_{j}(z\_{l}) \mid \mathcal{O}') p(\mathcal{O}') \\ &= p(a\_{l} < y \le a\_{l+1} \mid \mathcal{O}') p(\mathcal{O}') \\ &= p(a\_{l} < \mathcal{F}^{\mathrm{T}} \mathcal{O} + e \le a\_{l+1} \mid \mathcal{O}') p(\mathcal{O}') \\ &= p(a\_{l} < \mathcal{F}^{\mathrm{T}}(\mathcal{O}' + \overline{\mathcal{O}}) + e \le a\_{l+1} \mid \mathcal{O}') p(\mathcal{O}') \\ &= p(a\_{l} \cdot \mathcal{F}^{\mathrm{T}} \overline{\mathcal{O}} \cdot \mathcal{F}^{\mathrm{T}} \mathcal{O}' < e \le a\_{l+1} \cdot \mathcal{F}^{\mathrm{T}} \overline{\mathcal{O}} \cdot \mathcal{F}^{\mathrm{T}} \mathcal{O}' \cup \mathcal{O}') p(\mathcal{O}') \\ &= p(a\_{l} \cdot \mathcal{F}^{\mathrm{T}} \overline{\mathcal{O}} \cdot \mathcal{F}^{\mathrm{T}} \mathcal{O}' < e \le a\_{l+1} \cdot \mathcal{F}^{\mathrm{T}} \overline{\mathcal{O}} \cdot \mathcal{F}^{\mathrm{T}} \mathcal{O}') p(\mathcal{O}') \end{aligned} \tag{19}$$

where the last equality is based on the fact that*θ*'( )*j* and *e*(*j*) are stochastically independent, then

$$\begin{split} & \int\_{\theta' \in \mathbb{R}^{n \times m}} \mathcal{O}' p(\theta', \mathcal{y}\_{\mathrm{q}}^{\*} = D\_{j}(z\_{l})) \mathrm{d}\theta' \\ &= \int\_{\theta' \in \mathbb{R}^{n \times m}} \mathcal{O}' p(a\_{l} \cdot \mathcal{F}^{\mathrm{T}} \overline{\mathcal{O}} \cdot \mathcal{F}^{\mathrm{T}} \mathcal{O}' < e \leq a\_{l+1} \cdot \mathcal{F}^{\mathrm{T}} \overline{\mathcal{O}} \cdot \mathcal{F}^{\mathrm{T}} \mathcal{O}' \rangle p(\theta') \mathrm{d}\theta' \\ &= \int\_{\theta' \in \mathbb{R}^{n \times m}} \mathcal{O}'(T(\frac{a\_{l+1} \cdot \mathcal{F}^{\mathrm{T}} \overline{\mathcal{O}} \cdot \mathcal{F}^{\mathrm{T}} \mathcal{O}'}{\sqrt{R}}) \cdot T(\frac{a\_{l} \cdot \mathcal{F}^{\mathrm{T}} \overline{\mathcal{O}} \cdot \mathcal{F}^{\mathrm{T}} \mathcal{O}'}{\sqrt{R}}) \mathrm{G}(\theta', 0, \varPi) \mathrm{d}\theta' \end{split} \tag{20}$$

where *T*(.) is the probability distribution function of standardized normalized distribution (note that, *T*(.) is different from the tail function defined in (Ribeiro et al., 2006; You et al., 2011)), *G*( ',0, ) *θ Π* denotes the probability density function which means that stochastic vector *θ*' is Gaussian with zero-mean and covariance *Π* . Following the similar line of argument as in (Ribeiro et al., 2006; You et al., 2011), we calculate the integral (20) and then obtain

$$\int\_{\boldsymbol{\mathcal{O}} \in \mathbb{R}^{n \times n}} \boldsymbol{\mathcal{O}} \boldsymbol{p} (\boldsymbol{\mathcal{O}}', \boldsymbol{y}\_{\rm q}^\* = \boldsymbol{D}\_{\boldsymbol{\mathcal{I}}}(\boldsymbol{z}\_{\boldsymbol{l}})) \mathbf{d} \boldsymbol{\mathcal{O}}' = (\boldsymbol{\phi}(\frac{\boldsymbol{a}\_{\boldsymbol{l}} - \boldsymbol{F}^{\mathrm{T}} \overline{\boldsymbol{\Theta}}}{\sqrt{\boldsymbol{F}^{\mathrm{T}} \boldsymbol{I} \boldsymbol{F} + \boldsymbol{R}}}) - \boldsymbol{\phi}(\frac{\boldsymbol{a}\_{\boldsymbol{l}+1} - \boldsymbol{F}^{\mathrm{T}} \overline{\boldsymbol{\Theta}}}{\sqrt{\boldsymbol{F}^{\mathrm{T}} \boldsymbol{I} \boldsymbol{F} + \boldsymbol{R}}})) \frac{\boldsymbol{I} \boldsymbol{I} \boldsymbol{F}}{\sqrt{\boldsymbol{F}^{\mathrm{T}} \boldsymbol{I} \boldsymbol{F} + \boldsymbol{R}}} \tag{21}$$

where ( ) is the probability density function of standardized normalized distribution. Substitute (21) into (18), we have

$$\mathbb{E}(\mathcal{O}'(j)y\_q^\*(j)) = \sum\_{l=1}^N \frac{D\_j(z\_l)}{\sqrt{F^\top(j)\Pi(j)F(j) + R}} \langle \phi(\frac{a\_l \cdot F^\top(j)\overline{\theta}(j)}{\sqrt{F^\top(j)\Pi(j)F(j) + R}}) \cdot \phi(\frac{a\_{l+1} \cdot F^\top(j)\overline{\theta}(j)}{\sqrt{F^\top(j)\Pi(j)F(j) + R}}) |\varPi(j)F(j). \text{(22)}$$

Combining (22) with (17) and (14), we get that T \* \* q q *g* E{( ( ) - ( ))( ( ) - ( ))} 0 *θ k θ k yj yj* is equivalent to

$$\begin{split} \mathscr{S} \sum\_{l=1}^{N} \frac{D\_{j}(\mathbf{z}\_{l})}{\sqrt{\mathbf{F}^{\mathsf{T}}(j)\Pi(j)\mathbf{F}(j) + \mathbf{R}}} & (\boldsymbol{\phi}(\frac{\boldsymbol{a}\_{l} \cdot \mathbf{F}^{\mathsf{T}}(j)\overline{\boldsymbol{\theta}}(j)}{\sqrt{\mathbf{F}^{\mathsf{T}}(j)\Pi(j)\mathbf{F}(j) + \mathbf{R}}}) \cdot \boldsymbol{\phi}(\frac{\boldsymbol{a}\_{l+1} \cdot \mathbf{F}^{\mathsf{T}}(j)\overline{\boldsymbol{\theta}}(j)}{\sqrt{\mathbf{F}^{\mathsf{T}}(j)\Pi(j)\mathbf{F}(j) + \mathbf{R}}}) | \boldsymbol{A}^{k \cdot j} \cdot \Pi(j)\mathbf{F}(j) \neq \boldsymbol{0}, \\ & \exists j \leq k, \forall k \geq n + m - 1. \end{split} \tag{23}$$

Let

262 Stochastic Modeling and Control

Based on the Bayesian law,

where *θ*'( ) = ( ) - ( ) *j θ j θ j* . Let ' '( ) *θ θ j* , \* \*

+

*n m*

'

**R**

**R**

**R**

*θ*

'

*θ*

'

*θ*

+

Substitute (21) into (18), we have

*l*

*θ*

where

 **R** +

*n m*

+

*n m*

\* q \* q

( ', = ( ))

*p θ y Dz*

= ( = ( )| ') ( ') = ( < | ') ( ')

*py D z θ p θ pa y a θ p θ*

*j l*

*l l*

T

\* q

*θ p θ y Dz θ*

' ( ', = ( ))d '

+1

*j l*

*l l*

+1 T

= ( < + | ')p( ')

*pa F θ e a θ θ*

*l l l l l l*

*j l*

*l l*

*l l*

(Ribeiro et al., 2006; You et al., 2011), we calculate the integral (20) and then obtain

\* +1 <sup>q</sup> TT T =1

*<sup>N</sup> j l l l*

\* +1 <sup>q</sup> T TT '


( ) - () () - () () E{ '( ) ( ) = ( ( ) - ( )) ( ) ( ).

*D z aFj <sup>θ</sup> j a Fj <sup>θ</sup> <sup>j</sup> <sup>θ</sup> jy j <sup>Π</sup> jFj F j Π jFj R F j Π jFj R F j Π jFj R*

( ) ( ) ( )+ ( ) ( ) ( )+ ( ) ( ) ( )+

*a F <sup>θ</sup> a F <sup>θ</sup> <sup>Π</sup><sup>F</sup> <sup>θ</sup> <sup>p</sup> <sup>θ</sup> y Dz <sup>θ</sup>*

*j l*

time-variant variables (e.g. *F*(*j*), *Π*( )*j* ) is omitted for notational simplicity, then

*N*

*l θ N*

+

*n m*

*θ jy j θ y θ Dz p θ y Dz θ*

E{ '( ) ( )} E{ ' } = ' ( ) ( ', = ( ))d '

\* \* \* q q q =1 '

=1 '

*l θ*

+1

= ( < ( ' + ) + | ') ( ')

*pa F θ θ e a θ p θ*

+1 T T T T +1 T T T T +1

= ( - - ' < - - '| ') ( ') = ( - - ' < - - ') ( ')

where the last equality is based on the fact that*θ*'( )*j* and *e*(*j*) are stochastically independent, then

T T T T +1

*θ pa F θ F θ ea F θ F θ p θ θ*

= ' ( - - ' < - - ') ( ')d '

TT TT

*a F <sup>θ</sup> <sup>F</sup> <sup>θ</sup> a F <sup>θ</sup> <sup>F</sup> <sup>θ</sup> <sup>θ</sup> T TG θ Πθ R R*

T T

T T

 

 

*F ΠFR F ΠFR F ΠF R*

*l l*

( ) is the probability density function of standardized normalized distribution.


where *T*(.) is the probability distribution function of standardized normalized distribution (note that, *T*(.) is different from the tail function defined in (Ribeiro et al., 2006; You et al., 2011)), *G*( ',0, ) *θ Π* denotes the probability density function which means that stochastic vector *θ*' is Gaussian with zero-mean and covariance *Π* . Following the similar line of argument as in

 

*pa F θ F θ ea F θ F θ θ p θ pa F θ F θ ea F θ F θ p θ*

= ( ) ' ( ', = ( ))d '

 **R**

+

**R**

*n m*

*j l j l*

*D z θ p θ y Dz θ*

q q *y y j* ( ) , and the time variable *j* of other relevant

(18)

(19)

(20)

(21)

(22)

\* q

*j l j l*

$$\psi(j) \triangleq \sum\_{l=1}^{N} \frac{D\_j(\mathbf{z}\_l)}{\sqrt{\mathbf{F}^T(j)\Pi(j)\mathbf{F}(j) + R}} \langle \phi(\frac{\mathbf{a}\_l \cdot \mathbf{F}^T(j)\overline{\theta}(j)}{\sqrt{\mathbf{F}^T(j)\Pi(j)\mathbf{F}(j) + R}}) \cdot \phi(\frac{\mathbf{a}\_{l+1} \cdot \mathbf{F}^T(j)\overline{\theta}(j)}{\sqrt{\mathbf{F}^T(j)\Pi(j)\mathbf{F}(j) + R}}) \rangle \tag{24}$$

Then (23) is equivalent to that

$$\mathcal{W}\_k^{\text{id}\neq} = \sum\_{j=k}^0 \psi^2(j) A^{k \cdot j} \Pi(j) F(j) F^\top(j) \Pi(j) (A^{k \cdot j})^\top, \forall k \ge n+m-1 \tag{25}$$

has full rank. We conclude the above analysis as follows.

**Theorem 1**: The quantized linear system with Gauss-Markov parameters (6)(7)(8) is parameter identifiable, if and only if

$$
\tau\_{\text{max}} k \,\mathcal{W}\_k^{\text{id}\text{q}} = n + m\_\prime \,\forall k \ge n + m \text{ - 1} \tag{26}
$$

#### **Remark 2**:


$$\psi(j) \triangleq \sum\_{l=1}^{N} \frac{D\_j(z\_l(j))}{\sqrt{F^\top(j)\Pi(j)F(j) + R}} \langle \phi(\frac{a\_l(j) \cdot F^\top(j)\overline{\theta}(j)}{\sqrt{F^\top(j)\Pi(j)F(j) + R}}) \cdot \phi(\frac{a\_{l+1}(j) \cdot F^\top(j)\overline{\theta}(j)}{\sqrt{F^\top(j)\Pi(j)F(j) + R}}) \rangle \tag{27}$$

#### 264 Stochastic Modeling and Control

4. Condition (26) is equivalent to that the matrix sequence - { ( ) ( ) ( ), } *k j ψ j A Π jFj j k* has column rank *n*+*m*. - { ( ) ( ) ( ), } *k j <sup>ψ</sup> j A <sup>Π</sup> jFj j k* can be decomposed as - { ( ) ( ) ( ), } { (0), (1), , ( )} *k j <sup>ψ</sup> j A <sup>Π</sup> j F j j k diag ψψ ψ <sup>k</sup>* , where *diag*{ } denotes diagonal matrix. Hence the parameter identifiability of the original system can be preserved if *ψ*( ) 0, = 0,1,2, *j j* . Especially, condition (26) is equivalent to Lemma 1 for (6). Suppose we can design a time-variant quantizer as follows

$$\begin{aligned} D\_j(\mathbf{z}\_l(j)) &= c\_l \sqrt{\mathbf{F}^T(j)\Pi(j)F(j) + R}, \; l = 1, 2, \cdots, N, \\ a\_l(j) &= d\_l \sqrt{\mathbf{F}^T(j)\Pi(j)F(j) + R} + \mathbf{F}^T(j)\overline{\partial}(j), \; l = 1, 2, \cdots, N + 1, \; j = 0, 1, 2, \cdots, k, \end{aligned} \tag{28}$$

where *<sup>l</sup> c* and *<sup>l</sup> d* are constants which make

$$
\psi(j) = \sum\_{l=1}^{N} c\_1(\phi(d\_l) \cdot \phi(d\_{l+1})) \neq 0 \tag{29}
$$

Identifiability of Quantized Linear Systems 265

(33)

(34)

,where <sup>Δ</sup> +1 = - *ll l a a* , *l N* = 1,2, , , let <sup>T</sup> () - () () *l l aj a F j <sup>θ</sup> <sup>j</sup>* , then <sup>Δ</sup> +1 = ( ) - ( ), *ll l a j aj*

*l jl zj Dz F j θ j* . Consider the convergence property of the

+1

2

Let Δ = sup Δ*<sup>l</sup> 1kN*

Let ( ) = , *\**

Gramian id *Wk* .

**5. Simulation** 

Matlab:

<sup>T</sup> *F j* ( ) ( ) ( )+ *Π jFj R* , <sup>T</sup> ( ) ( )- ( ) ( ) *\**

*N N <sup>l</sup>*

 

*<sup>l</sup> z j <sup>r</sup>* then d () d= ,

Gramian idq *Wk* when <sup>Δ</sup> <sup>0</sup> . Note that *<sup>N</sup>* when <sup>Δ</sup> <sup>0</sup> , and then

=1

*N \**

*\**

*z j*

*\**

() () () lim ( ) = lim ( ( ) - ( ))

*zj aj a j <sup>ψ</sup> <sup>j</sup>*

( ) ( ) a ( )+ <sup>Δ</sup> = lim ( ( ) - ( ))

*l l*

( ) <sup>Δ</sup> <sup>0</sup> <sup>=</sup> =1

*z j <sup>s</sup> s*

*a j <sup>s</sup> <sup>l</sup>*

*<sup>l</sup> <sup>l</sup>*

2

<sup>1</sup> ( ) exp{- ( ) } ( ) <sup>2</sup> <sup>Δ</sup> = lim ( )

*z j*

( ) ( )+ <sup>Δ</sup> ( )- ( ) ( ) <sup>Δ</sup> = lim .(- ) <sup>Δ</sup>

*ll l*

*zj aj j*

( ) <sup>d</sup> <sup>Δ</sup> m ( )| (- ) <sup>d</sup>

*<sup>l</sup> \**

*l l*

2

*<sup>e</sup> <sup>ψ</sup> jr r <sup>π</sup>* **R**

**Remark 3**: Equation (34) implies the convergence of Theorem 1 to Lemma 1 when Δ 0 , i.e. the quantized identifiability Gramian idq *Wk* converges to the unquantized identifiability

In order to illustrate our main conclusion, the following system is simulated with the tool of

*y*( ) = ( ) ( - 1) + ( ) *k bkuk ek*

where *b k*( ) is the parameter to be identified, and can be modeled as a Gauss-Markov


*l l ll*

 

 

*l ll \* l l*

*aj aj*

 

 

2

2

*π*

*l l*

*\* <sup>l</sup> \* \* l l*

*z j z j z j π*

 

2 - <sup>2</sup> <sup>2</sup> lim ( ) = d = 1.

*r*

2

*\**

*z j*

<sup>Δ</sup> <sup>0</sup> =1

*l*

*l*

<sup>Δ</sup> <sup>0</sup> =1

<sup>Δ</sup> <sup>0</sup> =1

*l*

 **R**

*\* <sup>l</sup> z j <sup>r</sup>*

process. Then the system model can be transformed to

*l*

*N*

2

<sup>1</sup> ( ) exp{- ( ) } ( ) <sup>2</sup> d () =( )

*l*

= li

be the same for every *j*, thus,

$$\mathcal{W}\_k^{\text{idcl}} = \left(\sum\_{l=1}^N c\_l(\phi(d\_l) \cdot \phi(d\_{l+1}))\right)^2 \sum\_{j=k}^0 \psi^2(j) A^{k-j} \Pi(j) F(j) F^\top(j) \Pi(j) (A^{k-j})^\top. \tag{30}$$

By comparing (30) with (10), we can find that such a time-variant quantizer does not change the parameter identifiability of the original system if and only if (29) is satisfied;

5. The parameter identifiability of the system can be preserved even if the quantization level is low as long as the quantizer is designed reasonably. Especially, when quantization level = 2 *N* , set 1*c* = -1 , 2*c* = 1 , 1 *d* = - , 2 *d* = 0 , 3 *d* = + in the formula (28), then *ψ*( ) 2 / , = 0,1,2, *j π j* , namely a coarse quantizer of 1 bit can preserve the parameter identifiability of the original system.

#### **4. Convergence analysis**

In this section, we discuss the convergence property of the Gramian in Theorem 1, i.e., the convergence property of *ψ*( ), = 0,1,2, *j j* .

We know that

$$\sum\_{l=1}^{N} \frac{F^{\mathrm{T}}(\boldsymbol{j})\overline{\boldsymbol{\Theta}}(\boldsymbol{j})}{\sqrt{F^{\mathrm{T}}(\boldsymbol{j})\boldsymbol{\Pi}(\boldsymbol{j})F(\boldsymbol{j}) + R}} \phi(\frac{a\_{l} \cdot F^{\mathrm{T}}(\boldsymbol{j})\overline{\boldsymbol{\Theta}}(\boldsymbol{j})}{\sqrt{F^{\mathrm{T}}(\boldsymbol{j})\boldsymbol{\Pi}(\boldsymbol{j})F(\boldsymbol{j}) + R}}) \cdot \phi(\frac{a\_{l+1} \cdot F^{\mathrm{T}}(\boldsymbol{j})\overline{\boldsymbol{\Theta}}(\boldsymbol{j})}{\sqrt{F^{\mathrm{T}}(\boldsymbol{j})\boldsymbol{\Pi}(\boldsymbol{j})F(\boldsymbol{j}) + R}})) = 0 \tag{31}$$

by the property of( ) , then *ψ*( )*j* can be re-expressed as

$$\psi(j) = \sum\_{l=1}^{N} \frac{D\_j(\mathbf{z}\_l) \cdot F^\mathrm{T}(j)\overline{\theta}(j)}{\sqrt{F^\mathrm{T}(j)}\Pi(j)F(j) + R} \phi(\frac{a\_l \cdot F^\mathrm{T}(j)\overline{\theta}(j)}{\sqrt{F^\mathrm{T}(j)}\Pi(j)F(j) + R}) \cdot \phi(\frac{a\_{l+1} \cdot F^\mathrm{T}(j)\overline{\theta}(j)}{\sqrt{F^\mathrm{T}(j)}\Pi(j)F(j) + R})\tag{32}$$

Let Δ = sup Δ*<sup>l</sup> 1kN* ,where <sup>Δ</sup> +1 = - *ll l a a* , *l N* = 1,2, , , let <sup>T</sup> () - () () *l l aj a F j <sup>θ</sup> <sup>j</sup>* , then <sup>Δ</sup> +1 = ( ) - ( ), *ll l a j aj* <sup>T</sup> *F j* ( ) ( ) ( )+ *Π jFj R* , <sup>T</sup> ( ) ( )- ( ) ( ) *\* l jl zj Dz F j θ j* . Consider the convergence property of the Gramian idq *Wk* when <sup>Δ</sup> <sup>0</sup> . Note that *<sup>N</sup>* when <sup>Δ</sup> <sup>0</sup> , and then

 +1 =1 <sup>Δ</sup> <sup>0</sup> =1 <sup>Δ</sup> <sup>0</sup> =1 () () () lim ( ) = lim ( ( ) - ( )) ( ) ( ) a ( )+ <sup>Δ</sup> = lim ( ( ) - ( )) ( ) ( )+ <sup>Δ</sup> ( )- ( ) ( ) <sup>Δ</sup> = lim .(- ) <sup>Δ</sup> - = li *l l N \* ll l N N <sup>l</sup> \* l l ll l l ll \* l l l l zj aj a j <sup>ψ</sup> <sup>j</sup> zj aj j aj aj z j* ( ) <sup>Δ</sup> <sup>0</sup> <sup>=</sup> =1 2 2 <sup>Δ</sup> <sup>0</sup> =1 2 2 ( ) <sup>d</sup> <sup>Δ</sup> m ( )| (- ) <sup>d</sup> <sup>1</sup> ( ) exp{- ( ) } ( ) <sup>2</sup> <sup>Δ</sup> = lim ( ) 2 <sup>1</sup> ( ) exp{- ( ) } ( ) <sup>2</sup> d () =( ) 2 *<sup>l</sup> <sup>l</sup> l \* l l a j <sup>s</sup> <sup>l</sup> \* <sup>l</sup> \* l l l \* <sup>l</sup> \* \* l l z j <sup>s</sup> s z j z j π z j z j z j π* **R** (33) Let ( ) = , *\* <sup>l</sup> z j <sup>r</sup>* then d () d= , *\* <sup>l</sup> z j <sup>r</sup>* 2 *r*

$$\lim\_{N \to \infty} \psi(j) = \int\_{\mathbb{R}} r^2 \frac{e^{\frac{\cdot}{2}}}{\sqrt{2\pi}} d\mathbf{r} = 1. \tag{34}$$

**Remark 3**: Equation (34) implies the convergence of Theorem 1 to Lemma 1 when Δ 0 , i.e. the quantized identifiability Gramian idq *Wk* converges to the unquantized identifiability Gramian id *Wk* .

#### **5. Simulation**

264 Stochastic Modeling and Control

4. Condition (26) is equivalent to that the matrix sequence - { ( ) ( ) ( ), } *k j ψ j A Π jFj j k* has

Suppose we can design a time-variant quantizer as follows

T T ( ( )) = ( ) ( ) ( ) + , = 1,2, , ,

*Dzj c F j Π jFj R l N*

T

*jl l*

where *<sup>l</sup> c* and *<sup>l</sup> d* are constants which make

*N*

*k ll l*

*W cd d* 

parameter identifiability of the original system.

*l l*

be the same for every *j*, thus,

**4. Convergence analysis** 

We know that

*N*

*l*

by the property of

convergence property of *ψ*( ), = 0,1,2, *j j* .

*l*

column rank *n*+*m*. - { ( ) ( ) ( ), } *k j <sup>ψ</sup> j A <sup>Π</sup> jFj j k* can be decomposed as - { ( ) ( ) ( ), } { (0), (1), , ( )} *k j <sup>ψ</sup> j A <sup>Π</sup> j F j j k diag ψψ ψ <sup>k</sup>* , where *diag*{ } denotes diagonal matrix.

Hence the parameter identifiability of the original system can be preserved if *ψ*( ) 0, = 0,1,2, *j j* . Especially, condition (26) is equivalent to Lemma 1 for (6).

+1

*k j k j*

*ψ j A Π jFjF j Π j A* (30)

+1

  +1

(29)

(28)

(31)

(32)

( ) = ( ) ( ) ( ) + + ( ) ( ), = 1,2, , + 1 , = 0,1,2, , ,

( ) = ( ( ) - ( )) 0

*ll l*

 

= ( ( ( ) - ( ))) ( ) ( ) ( ) ( ) ( )( ) .

By comparing (30) with (10), we can find that such a time-variant quantizer does not change

5. The parameter identifiability of the system can be preserved even if the quantization level is low as long as the quantizer is designed reasonably. Especially, when quantization level = 2 *N* , set 1*c* = -1 , 2*c* = 1 , 1 *d* = - , 2 *d* = 0 , 3 *d* = + in the formula (28), then *ψ*( ) 2 / , = 0,1,2, *j π j* , namely a coarse quantizer of 1 bit can preserve the

In this section, we discuss the convergence property of the Gramian in Theorem 1, i.e., the

T T T

( ) ( ) ( )+ ( ) ( ) ( )+ ( ) ( ) ( )+

( )- ( ) ( ) - () () - () () ( ) = () - ( ))

*F j θ j aFj θ j a Fj θ j F j Π jFj R F j Π jFj R F j Π jFj R*

() () - () () - () () ( ) - ( )) <sup>=</sup> <sup>0</sup>

 

*l l*

T T T

( ) ( ) ( )+ ( ) ( ) ( )+ ( ) ( ) ( )+

*F j Π jFj R F j Π jFj R F j Π jFj R*

TT T =1

( ) , then *ψ*( )*j* can be re-expressed as

TT T =1

*<sup>N</sup> j l l l*

*Dz F j <sup>θ</sup> <sup>j</sup> aFj <sup>θ</sup> j a Fj <sup>θ</sup> <sup>j</sup> <sup>ψ</sup> <sup>j</sup>*

*aj d F j Π jFj R F j θ jl N j k*

=1

the parameter identifiability of the original system if and only if (29) is satisfied;

0 idq 2 2 - T - T

*l <sup>ψ</sup> j cd d* 

+1

=1 =

 

*l j k*

*N*

In order to illustrate our main conclusion, the following system is simulated with the tool of Matlab:

$$y(k) = b(k)\mu(k \cdot 1) + e(k)$$

where *b k*( ) is the parameter to be identified, and can be modeled as a Gauss-Markov process. Then the system model can be transformed to

$$\begin{cases} \Theta(k+1) = a\Theta(k) + \varpi(k) \\ y(k) = F^\top(k)\Theta(k) + e(k) \end{cases}$$

Identifiability of Quantized Linear Systems 267

actual value estimate

In Fig. 1(a) and Fig. 2(a), actual values of parameter are denoted by solid lines and the estimates are denoted by dotted lines. Estimation errors are shown in Fig. 1(b) and Fig. 2(b). Fig. 1 and Fig. 2 show that the estimate can track the real value of the parameter when the outputs are quantized coarsely. The curves of prior error entropy and posterior error

<sup>1</sup> ( ) = ln2 + ln 2 2

where *<sup>n</sup> x***R** is a Gaussian vector with covariance *C*, | | denotes determinant. For quantized systems, the probability distribution of estimation error *θ*( ) *k* is unknown, but is supposed to make the entropy of *θ*( ) *k* maximal according to "maximal entropy principle" of Jaynes (Jaynes, 1957), namely, the uncertainty of *θ*( ) *k* is supposed to be maximal in the situation of lack of prior information, hence *θ*( ) *k* is assumed to be Gaussian, and thus (35) can be adopted to calculate the entropy of *θ*( ) *k* in this simulation. We can observe from Fig. 1(c) and Fig. 2(c) that the posterior error entropy is strictly smaller than prior error entropy from the initial time instant. This indicates that this quantized system is parameter identifiable, and these observations consist with our analysis mentioned above perfectly. Besides, we can observe that the estimation error when quantization level = 2 *N* is greater than that in the case of quantization level = 4 *N* though the system is parameter identifiable in both of the two quantization cases. This shows that systems with different quantizers lead to different estimation precision, though all of them are parameter identifiable when rational

50 55 60 65 70 75 80 85 90 95 100

k

(a)

*<sup>n</sup> H x <sup>π</sup>e C* (35)

entropy are shown in Fig. 1(c) and Fig. 2(c). The entropy is calculated by

quantizers are used.




0

Actual value and Estimate of b(k)

1

2

3

where *θ*( )= ( ) *k bk* , *a*=0.5. *e k*( ) , *w k*( ) and *θ*(0) are mutually statistically uncorrelated, their covariance are *Q*=1, *R*=0.1, *Π*(0) = 1 respectively, and the mean of *θ*(0) is *θ* = 1 . Here we set *Fk uk k* ( ) = ( - 1) = 2sin( ) + 3 as the assumed system input (i.e., the control signal, which can be considered to be generated, for example, by the adaptive controller), where the additive term "3" plays the role of avoiding the problem of "turn-off" (Astrom & Wittenmark, 1994).

To do the illustrative simulation, an optimal filter is required though the analysis about parameter identifiability is independent of the estimator used. The discussed linear system with Gauss-Markov parameter is transformed to state space model, and then the problem of parameter identification can be treated as states estimation. A number of quantized state estimators have been proposed by scholars in various areas, and we choose the Gaussian fit algorithm (Curry, 1970) as the filter in this section for that this filter which bases on the Gaussian assumption is near optimal and convenient to be implemented. Note that, in this simulated model, *F k*( ) is defined by *u k*( - 1) completely, so it is known at the channel receiver; however, in general model (1) ( ), = 1,2, , *<sup>i</sup> aki n* and ( ), = 1,2, , *<sup>i</sup> bki m* are to be identified, then *F k*( ) is defined by *uk uk m* ( - 1) ( - ) and - ( - 1) - ( - ) *y k yk n* jointly. So in general, the quantized signals q q *y* ( - 1), , ( - ) *k y kn* , instead of the actual outputs *y*( - 1), , ( - ) *k yk n* are received at the channel receiver, thus *y*( - 1), , ( - ) *k yk n* in *F k*( ) should be replaced by their estimates.

The analysis about parameter identifiability of quantized systems is suitable for any rational quantizer. Here the Max-Lloyd quantizer (Proakis, 2001) generally adopted in areas of communication and signal processing is employed. In the following statement, cases of quantization level = 4 *N* and = 2 *N* in (7) are simulated, respectively. The thresholds of the 4 level Max-Lloyd quantizer are {–∞, –0.9816, 0, 0.9816, +∞} and the outputs of the quantizer are {–1.51, –0.4528, 0.4528, 1.51} when the signal to be quantized is standardized normally distributed. In the case of 2 level quantizer, the thresholds are {– ∞, 0, +∞} and the outputs of the quantizer are {–0.7979, 0.7979}. Hence the thresholds of the time-variant quantizers with 4 and 2 levels are respectively ( ) σ × *y k* {–∞, –0.9816, 0, 0.9816, +∞}+ ( ) × *y k E* {1, 1, 1, 1, 1} and ( ) σ × *y k* {–∞, 0, +∞}+ ( ) × *y k E* {1, 1, 1}, the outputs of the quantizers are ( ) σ × *y k* {–1.51, –0.4528, 0.4528, 1.51}+ ( ) × *y k E* {1, 1, 1, 1} and ( ) σ × *y k* {–0.7979, 0.7979}+ ( ) × *y k E* {1, 1}, where *y k*( ) *E* and ( ) σ*y k* are the mean and standard deviation of the output *y*(*k*) respectively.

It is obvious that the above model is parameter identifiable by Lemma 1 when it is unquantized. We get *ψ*( ) 0.8823 *j* by calculating the weight *ψ*( )*j* in equation (27) when the 4 level time-variant Max-Lloyd quantizer is used and *ψ*( ) 0.6366 *j* when quantization level *N* = 2 . Hence the parameter identifiability will not be changed by the quantization according to Remark 2, i.e. the quantized system is still parameter identifiable, theoretically. The simulation results of the quantized system shown in Fig. 1 ( = 4 *N* ) and Fig. 2 ( = 2 *N* ) illustrate the above conclusion.

In Fig. 1(a) and Fig. 2(a), actual values of parameter are denoted by solid lines and the estimates are denoted by dotted lines. Estimation errors are shown in Fig. 1(b) and Fig. 2(b). Fig. 1 and Fig. 2 show that the estimate can track the real value of the parameter when the outputs are quantized coarsely. The curves of prior error entropy and posterior error entropy are shown in Fig. 1(c) and Fig. 2(c). The entropy is calculated by

266 Stochastic Modeling and Control

Wittenmark, 1994).

should be replaced by their estimates.

output *y*(*k*) respectively.

illustrate the above conclusion.

T ( + 1) = ( ) + ( ) ( )= ( ) ( )+ ( ) *θ k aθ k wk yk F k θ k ek*

where *θ*( )= ( ) *k bk* , *a*=0.5. *e k*( ) , *w k*( ) and *θ*(0) are mutually statistically uncorrelated, their covariance are *Q*=1, *R*=0.1, *Π*(0) = 1 respectively, and the mean of *θ*(0) is *θ* = 1 . Here we set *Fk uk k* ( ) = ( - 1) = 2sin( ) + 3 as the assumed system input (i.e., the control signal, which can be considered to be generated, for example, by the adaptive controller), where the additive term "3" plays the role of avoiding the problem of "turn-off" (Astrom &

To do the illustrative simulation, an optimal filter is required though the analysis about parameter identifiability is independent of the estimator used. The discussed linear system with Gauss-Markov parameter is transformed to state space model, and then the problem of parameter identification can be treated as states estimation. A number of quantized state estimators have been proposed by scholars in various areas, and we choose the Gaussian fit algorithm (Curry, 1970) as the filter in this section for that this filter which bases on the Gaussian assumption is near optimal and convenient to be implemented. Note that, in this simulated model, *F k*( ) is defined by *u k*( - 1) completely, so it is known at the channel receiver; however, in general model (1) ( ), = 1,2, , *<sup>i</sup> aki n* and ( ), = 1,2, , *<sup>i</sup> bki m* are to be identified, then *F k*( ) is defined by *uk uk m* ( - 1) ( - ) and - ( - 1) - ( - ) *y k yk n* jointly. So in general, the quantized signals q q *y* ( - 1), , ( - ) *k y kn* , instead of the actual outputs *y*( - 1), , ( - ) *k yk n* are received at the channel receiver, thus *y*( - 1), , ( - ) *k yk n* in *F k*( )

The analysis about parameter identifiability of quantized systems is suitable for any rational quantizer. Here the Max-Lloyd quantizer (Proakis, 2001) generally adopted in areas of communication and signal processing is employed. In the following statement, cases of quantization level = 4 *N* and = 2 *N* in (7) are simulated, respectively. The thresholds of the 4 level Max-Lloyd quantizer are {–∞, –0.9816, 0, 0.9816, +∞} and the outputs of the quantizer are {–1.51, –0.4528, 0.4528, 1.51} when the signal to be quantized is standardized normally distributed. In the case of 2 level quantizer, the thresholds are {– ∞, 0, +∞} and the outputs of the quantizer are {–0.7979, 0.7979}. Hence the thresholds of the time-variant quantizers with 4 and 2 levels are respectively ( ) σ × *y k* {–∞, –0.9816, 0, 0.9816, +∞}+ ( ) × *y k E* {1, 1, 1, 1, 1} and ( ) σ × *y k* {–∞, 0, +∞}+ ( ) × *y k E* {1, 1, 1}, the outputs of the quantizers are ( ) σ × *y k* {–1.51, –0.4528, 0.4528, 1.51}+ ( ) × *y k E* {1, 1, 1, 1} and ( ) σ × *y k* {–0.7979, 0.7979}+ ( ) × *y k E* {1, 1}, where *y k*( ) *E* and ( ) σ*y k* are the mean and standard deviation of the

It is obvious that the above model is parameter identifiable by Lemma 1 when it is unquantized. We get *ψ*( ) 0.8823 *j* by calculating the weight *ψ*( )*j* in equation (27) when the 4 level time-variant Max-Lloyd quantizer is used and *ψ*( ) 0.6366 *j* when quantization level *N* = 2 . Hence the parameter identifiability will not be changed by the quantization according to Remark 2, i.e. the quantized system is still parameter identifiable, theoretically. The simulation results of the quantized system shown in Fig. 1 ( = 4 *N* ) and Fig. 2 ( = 2 *N* )

 

$$H(\mathbf{x}) = \frac{n}{2} \text{ln} 2\pi e + \frac{1}{2} \text{ln} \left| \mathbf{C} \right| \tag{35}$$

where *<sup>n</sup> x***R** is a Gaussian vector with covariance *C*, | | denotes determinant. For quantized systems, the probability distribution of estimation error *θ*( ) *k* is unknown, but is supposed to make the entropy of *θ*( ) *k* maximal according to "maximal entropy principle" of Jaynes (Jaynes, 1957), namely, the uncertainty of *θ*( ) *k* is supposed to be maximal in the situation of lack of prior information, hence *θ*( ) *k* is assumed to be Gaussian, and thus (35) can be adopted to calculate the entropy of *θ*( ) *k* in this simulation. We can observe from Fig. 1(c) and Fig. 2(c) that the posterior error entropy is strictly smaller than prior error entropy from the initial time instant. This indicates that this quantized system is parameter identifiable, and these observations consist with our analysis mentioned above perfectly. Besides, we can observe that the estimation error when quantization level = 2 *N* is greater than that in the case of quantization level = 4 *N* though the system is parameter identifiable in both of the two quantization cases. This shows that systems with different quantizers lead to different estimation precision, though all of them are parameter identifiable when rational quantizers are used.

Identifiability of Quantized Linear Systems 269

actual value estimate

(a)

(b) 50 55 60 65 70 75 80 85 90 95 100

k

k

50 55 60 65 70 75 80 85 90 95 100







Estimation error of b(k)

0

0.5

1

1.5

2



0

Actual value and Estimate of b(k)

1

2

3

4

**Figure 1.** (a) Actual value and estimate of b(k), *N*=4, (b) Estimation error of b(k), *N*=4, (c) Prior and posterior error entropy, *N*=4

268 Stochastic Modeling and Control

1.5




0

Estimation error of b(k)

0.5

1

**Figure 1.** (a) Actual value and estimate of b(k), *N*=4, (b) Estimation error of b(k), *N*=4,

(c)

k

0 10 20 30 40 50 60 70 80 90 100

(b)

k

50 55 60 65 70 75 80 85 90 95 100

Prior error entropy Posterior error entropy

(c) Prior and posterior error entropy, *N*=4

0

0.2

0.4

0.6

0.8

Prior and posterior error entropy

1

1.2

1.4

1.6

Identifiability of Quantized Linear Systems 271

**Author details** 

**Acknowledgement** 

61174063 & 60736021.

**7. References** 

00189286.

1114, ISSN 00051179.

171-190, ISSN 0031899X.

ISBN 0-07-232111-3, New York, NY.

ISSN 0018-9286.

Corresponding Author

 \*

*China Mobile Group, Zhejiang Co., Ltd. Huzhou Branch, Huzhou, China* 

Publishing Company, ISBN 0201558661, Boston, MA, USA.

& Sons, Inc., ISBN 10-0-471-24195-4, New Jersey.

vol. 21, no. 4, (August 2010), pp. 337-370, ISSN 09324194.

vol. 44, no. 6, (October 1998), pp. 2325-2383, ISSN 00189448.

M.I.T Press, ISBN 0-262-53216-6, London.

*State Key Laboratory of Industrial Control Technology, Institute of Industrial Process Control, Department of Control Science and Engineering, Zhejiang University, Hangzhou, China* 

This work was supported by the Natural Science Foundation of China under Grand No.

Astrom, K. J. & Wittenmark, B. (1994). *Adaptive Control* (2nd edition), Addison-Wesley

Baram, Y. & Kailath T. (1988). Estimability and Regulability of Linear Systems. *IEEE Transactions on Automatic Control*, vol. 33, no. 12, (December 1988), pp. 1116-1121, ISSN

Cover, T. M. & Thomas, J. A. (2006). *Elements of Information Theory* (2nd edition). John Wiley

Curry, R. E. (1970). *Estimation and Control with Quantized Measurements* (1st edition). The

De Persis, C. & Mazenc, F. (2010). Stability of Quantized Time-Delay Nonlinear Systems: A Lyapunov-Krasowskii-Functional Approach. *Mathematics of Control, Signals, & Systems*,

Durgaryan, I. S. & Pashchenko, F. F. (2001). Identification of Objects by the Maximal Information Criterion. *Automation & Remote Control*, vol. 62, no. 7, (July 2001), pp. 1104-

Fu, M.-Y. & Xie, L.-H. (2005). The Sector Bound Approach to Quantized Feedback Control. *IEEE Transactions on Automatic Control*, vol. 50, no. 11, (November 2005), pp. 1698-1711,

Gray, R. M. & Neuhoff, D. L. (1998). Quantization. *IEEE Transactions on Information Theory*,

Jaynes, E. T. (1957). Information Theory and Statistical Mechanics, I & II. *Physical Review*, vol. 106, no. 4, (May 1957), pp. 620-630, ISSN 0031899X, vol. 108, no. 2, (October 1957), pp.

Proakis, J. G. (2001). *Digital Communications* (4th edition). The McGraw-Hill Companies,

Ying Shen

Hui Zhang\*

**Figure 2.** (a) Actual state and estimate of b(k), *N*=2, (b) Estimation error of b(k), *N*=2, (c) Prior and posterior error entropy, *N*=2

## **6. Conclusion**

This paper discusses the parameter identifiability of quantized linear systems with Gauss-Markov parameters from information theoretic point of view. The existing definition concerning this property is reviewed and new definition is proposed for quantized systems. Criterion function, the Gramian of parameter identifiability for quantized systems is analyzed based on the quantity of mutual information. The derived conclusions consist with our intuition very well and also provide us with intrinsic perspective for the quantizer design. The analysis shows that the Gramian of quantized systems converge to that of unquantized systems when the quantization intervals turn to zero, and a well designed quantizer can preserve the identifiability of the original system even if the quantizer is as coarse as one bit. The analytical analysis is verified by the illustrative simulation.
