*2.5.1 Definitions*

of *iN* at time *N* conditioned by the *n* last events, is practically equal to the probabil-

*p i*ð Þ *<sup>N</sup>*, *N*j *iN*�1, *N* � 1; … *in*, *N* � *n* ≈ *p i*ð Þ *<sup>N</sup>*, *N*j *iN*�1, *N* � 1; … *i*0, 0 when *n* ! ∞*:*

More precisely, for any *ε* > 0, there exists a positive integer *n* such that for any

where *sn* is the instantaneous entropy given by (14). In fact, let us write

*<sup>N</sup>* ð Þ¼ *iN p iN*, *N i*<sup>j</sup> *<sup>N</sup>*�1, *<sup>N</sup>* � 1; *::* … ; *iN*�*<sup>n</sup>*, *<sup>N</sup>* � *<sup>n</sup>*<sup>Þ</sup> <sup>¼</sup> *<sup>μ</sup> <sup>f</sup>* �*<sup>N</sup>iN <sup>f</sup>* �*N*þ<sup>1</sup>*iN*�<sup>1</sup><sup>∩</sup> … <sup>∩</sup>*<sup>f</sup>* �*N*þ*<sup>n</sup>iN*�*<sup>n</sup>*

For a given *n*, formula (23) allows one to define a new process *p*(*n*) from the original process *p*, which can be called "the approximate process of order *n*" of *p* (see Section 2.6). It results from (21) and from the stationarity of *p* that for any *ε* > 0, there is an integer *n*(*ε*) depending only on *ε*, such that for any integers *N*,

last right hand member of Eq. (22) is the average of this relative entropy on the past

The total variation distance *d*(*P*,*Q*) between two distributions *Pj* and *Qj* over the

of *N.* Because *sN*(*p*) decreases to a limit *s p*ð Þ when *N* ! ∞, it results that

*d P*ð Þ¼ , *<sup>Q</sup>* <sup>1</sup>

Then, the total variation distance *<sup>d</sup>*0, … *<sup>N</sup>*�<sup>1</sup> *<sup>Π</sup>N*, *<sup>Π</sup>*ð Þ *<sup>n</sup>*

*2.4.3 Convergence properties of the approximate process*

2 X *j*

given past trajectory between times 0 and *N*-1) is related to the relative entropy

h i � � <sup>2</sup> � �

Let us write *m* = *N-n* > 0. It follows [18] from (25) that for any fixed *m*, the total

*tends to* 0 *in probability* when *n* ! ∞

<sup>≤</sup> *<sup>d</sup>*0, … *<sup>N</sup>*�<sup>1</sup> *<sup>Π</sup>N*, *<sup>Π</sup>*ð Þ *<sup>n</sup>*

variation distance between the exact and the approximate probabilities

*<sup>Π</sup>N*ð Þ¼ *iN p iN*, *N i*<sup>j</sup> *<sup>N</sup>*�1, *<sup>N</sup>* � 1; *::* … *<sup>i</sup>*0, 0<sup>Þ</sup> <sup>¼</sup> *<sup>μ</sup> <sup>f</sup>* �*<sup>N</sup>iN <sup>f</sup>* �*N*þ<sup>1</sup>*iN*�<sup>1</sup><sup>∩</sup> … <sup>∩</sup>*i*<sup>0</sup>

0<*sN* <*sn* <*ε:* (21)

�

*<sup>μ</sup> <sup>i</sup>*0∩*<sup>f</sup>* �<sup>1</sup>ð Þ *<sup>i</sup>*<sup>1</sup> <sup>∩</sup> … <sup>∩</sup>*<sup>f</sup>* �*N*þ<sup>1</sup>ð Þ *iN*�<sup>1</sup> *<sup>S</sup>*0, … *<sup>N</sup>*�<sup>1</sup> *<sup>Π</sup><sup>N</sup> <sup>Π</sup>*ð Þ *<sup>n</sup>*

is the relative entropy of *Π<sup>N</sup>* with respect to *Π*ð Þ *<sup>n</sup>*

0<*δsn*ð Þ� *p sn* ð Þ� *p s p*ð Þ ≤ *ε* if *n*>*n*ð Þ*ε :* (25)

*P <sup>j</sup>* � *Q <sup>j</sup>* � � �

*N*

� �

*N* � � �

� � �; � (22)

� � �*:* �

(20)

(23)

<*ε:*

(24)

*<sup>N</sup>* : the

*<sup>N</sup>* (for a

*N* � � � � �

�*:* (26)

between *Π<sup>N</sup>* and *Π*ð Þ *<sup>n</sup>*

<*ε=*2 if *n*ð Þ*ε* <*n* < *N:* (27)

ity at time *N*, conditioned by the *whole past* down to time 0.

*Advances in Dynamical Systems Theory, Models, Algorithms and Applications*

*N* > *n*

*Π*ð Þ *<sup>n</sup>*

*n* > *n*(*ε*)

<sup>0</sup><*sn* ð Þ� *<sup>p</sup> sN*ð Þ¼ *<sup>p</sup>* <sup>X</sup>

where *<sup>S</sup>*0, … *<sup>N</sup>*�<sup>1</sup> *<sup>Π</sup>N*|*Π*ð Þ *<sup>n</sup>*

states *j* of a finite set (*j*) is

*<sup>d</sup>*0, … *<sup>N</sup>*�<sup>1</sup> *<sup>Π</sup>N*, *<sup>Π</sup>*ð Þ *<sup>n</sup>*

*<sup>d</sup>*0, … *<sup>m</sup>*þ*n*�<sup>1</sup> *<sup>Π</sup><sup>m</sup>*þ*<sup>n</sup>*, *<sup>Π</sup>*ð Þ *<sup>n</sup>*

**90**

D E � � <sup>2</sup>

[18, 31] and it can be concluded that

*N*

*m*þ*n* � �

*i*0, … *iN*�<sup>1</sup>

*N* � � �

i. *simplified definition*: a (discrete time) sequence of stochastic variables *Xn* is a martingale if for all *n*:

$$
\langle |X\_n| \rangle < \infty \quad \text{and} \quad \langle X\_{n+1} | X\_n, \dots X\_1 \rangle \quad = X\_n. \tag{30}
$$

where h i *X* denotes the average (mathematical expectation) of the stochastic variable *X*.

ii. *more generally* (see the general definition, for instance, in [20])

	- F <sup>n</sup> is an increasing sequence of *σ*-algebras extracted from F (F <sup>n</sup> ⊂ F *<sup>n</sup>* + 1 ⊂ … ⊂ F), and.
	- for all *n* ≥ 0, *Xn* is a stochastic variable defined on (Ω,F n, *P*),

the sequence *<sup>X</sup>*<sup>n</sup> is a martingale if h i j j *Xn* <sup>&</sup>lt; <sup>∞</sup> and *Xn*þ<sup>1</sup>jF*<sup>n</sup>* <sup>¼</sup> *Xn*.

#### *2.5.2 Convergence theorem for martingales*

Among the remarkable properties of martingales, the following convergence theorem holds [20, 21]:

If (*Xn*) is a positive martingale, the sequence *Xn* converges almost surely to a stochastic variable *X*.

So, for almost all trajectories *ω*, *Xn*(*ω*) ! *X*(*ω*) with probability 1 when *n* ! ∞. Stronger and more general results can be found in the references.
