**2.1 Markov modulated fluid flow (MMFF)**

Let ℑ be a state space that can partitioned into two sets: ℑ ¼ ℑ<sup>1</sup> ∪ ℑ2, with j j ℑ<sup>1</sup> ¼ *n*1, j j ℑ<sup>2</sup> ¼ *n*2, and j j ℑ ¼ *n*<sup>1</sup> þ *n*<sup>2</sup> ¼ *n:* Now, we introduce a modulating continuous time Markov chain (CTMC) fJ ð Þ*t* ; *t*>0g with that state ℑ, and a fluid process f g Fð Þ*t* ; *t*≥ 0 that is modulated as follows: whenever the Markov chain is J ðÞ¼ *t i* ∈ ℑ1, the fluid flow increases linearly at rate *ci* >0 and whenever it is in J ðÞ¼ *t j*∈ ℑ2, the fluid flow decreases linearly at rate *cj* <sup>&</sup>gt; <sup>0</sup> *cj* <sup>&</sup>lt;<sup>0</sup> *:* The two-dimensional stochastic process f g Fð Þ*t* , J ð Þ*t* , *t*≥0 is called a MMFF (Markov Modulated Fluid Flow) process. We denote by the infinitesimal generator matrix of J ð Þ*t* ; is given in a block form according to transitions between the sets ℑ*<sup>i</sup>* (*i* ¼ 1, 2). Let 1, <sup>2</sup> and be diagonal matrices as follows:

*Fluid Inventory Models under Markovian Environment DOI: http://dx.doi.org/10.5772/intechopen.104183*

$$\mathbb{C}\_{j} = \operatorname{diag} \left\{ |c\_{i}|, i \in \mathfrak{T}\_{j} \right\}, \quad j = \mathbf{1}, 2.$$

$$\mathbb{C} = \operatorname{diag} \left( \mathbb{C}\_{1}, \mathbb{C}\_{2} \right).$$

Let *τ*ð Þ¼ *x* inf ð Þ *t*>0, FðÞ¼ *t x* be the first passage time to level *x:* Let Ψð Þ *β* be an ð Þ *n*<sup>1</sup> � *n*<sup>2</sup> matrix whose *ij*th component is

$$[\Psi(\boldsymbol{\beta})]\_{\check{\boldsymbol{\eta}}} = E\left(e^{-\beta \boldsymbol{\tau}(\boldsymbol{0})}, \mathcal{J}(\boldsymbol{\tau}(\mathbf{0})) = j | \mathcal{F}(\mathbf{0}) = \mathbf{0}, \mathcal{J}(\mathbf{0}) = i\right) \quad i \in \mathfrak{F}\_1 \\ j \in \mathfrak{F}\_2,$$

which is the LST of *τ*ð Þ 0 restricted to the event that the fluid process hits level 0 in-phase *j*∈ ℑ<sup>2</sup> and given that Fð Þ¼ 0 0,J ð Þ¼ 0 *i*∈ ℑ1*:* In the literature, a few algorithms, including some quadratically convergent ones, were established for computing Ψð Þ *β* (and all other LSTs); see, e.g., Ahn and Ramaswami [36]. Let *<sup>τ</sup>*ð Þ *<sup>x</sup>*, *<sup>y</sup>* be the first passage time of <sup>F</sup> from level *<sup>x</sup>* to level *<sup>y</sup>*, and *<sup>b</sup> <sup>a</sup>τ*ð Þ *x*, *y* be the first passage time of F from level *x* to level *y* avoiding a visit below *a* or above *<sup>b</sup>* (for simplicity, we use *<sup>b</sup>τ*ð Þ� *<sup>x</sup>*, *<sup>y</sup> <sup>b</sup>* <sup>0</sup>*τ*ð Þ *<sup>x</sup>*, *<sup>y</sup>* and *<sup>a</sup>τ*ð Þ� *<sup>x</sup>*, *<sup>y</sup>* <sup>∞</sup> *<sup>a</sup> <sup>τ</sup>*ð Þ *<sup>x</sup>*, *<sup>y</sup>* ). Let ^*f x*ð Þ , *<sup>y</sup>*, *<sup>β</sup>* and *b a* ^*f x*ð Þ , *<sup>y</sup>*, *<sup>β</sup>* denote, respectively, the LST matrices of the joint distribution of the first passage times *<sup>τ</sup>*ð Þ *<sup>x</sup>*, *<sup>y</sup>* , *<sup>b</sup> <sup>a</sup>τ*ð Þ *x*, *y* and the state of the phase process at each first passage time.

An important variant of the fluid flow F, a *Reflected Fluid Flow,* is particularly useful in the analysis of our inventory level process. The reflected fluid flow F*<sup>r</sup>* is obtained by reversing the roles of the up and down environment states. Analogous, Ψ*r* ð Þ *β* is the matrix (of order ð Þ *n*<sup>2</sup> � *n*<sup>1</sup> ) whose ð Þ *i*, *j* component is the LST of the time to reach the level 0 for the process F*<sup>r</sup>* restricted to J *<sup>r</sup>* ð Þ¼ *τ*ð Þ 0 *j*∈ ℑ1, given that F*r* ð Þ¼ <sup>0</sup> 0 and <sup>J</sup> *<sup>r</sup>* ð Þ¼ <sup>0</sup> *<sup>i</sup>*<sup>∈</sup> <sup>ℑ</sup>2, where <sup>J</sup> *<sup>r</sup>* ð Þ*<sup>t</sup>* is the modulated state process for <sup>F</sup>*<sup>r</sup>* (we use notations ^*f r* ð Þ *<sup>x</sup>*, *<sup>y</sup>*, *<sup>β</sup>* and *<sup>b</sup> a* ^*f r* ð Þ *x*, *y*, *β* to denote quantities similar to those above defined for F*<sup>r</sup>* ).

All these matrices, for hitting times, that we will use are straightforward to evaluate once we have computed Ψð Þ *β :* **Table 1** displays the basic elements (matrices) for the derivation of these LSTs; the first three matrices are associated with to flow F, while the next three matrices are associated to the rate-reverse flow F*<sup>r</sup>* by interchanging the indices 1 and 2. The LST matrices and their sizes are given in **Table 2**; all matrices have nice probabilistic interpretations. For more details see Ramaswami [33], Ahn *et al*. [34], and Bean *et al.* [38].

