**4.2 The matrix** *e*�*β<sup>C</sup>* � �

Given the state J ð Þ *L* , the ON period and the OFF period are independent. Clearly, the ON period ends at state in ℑ<sup>1</sup> (at time *L*). However, at that point of time, the production stops, and thus, the state at which the OFF period starts can be either in ℑ<sup>1</sup> or in ℑ2*:* Similarly, the OFF period ends at state in ℑ<sup>2</sup> (at time *T*). However, since the production starts at that point, the ON period can start either in state in ℑ<sup>1</sup> or in ℑ2. This needs to be considered, particularly at switching times. Hence, at these switching times, the phases are labeled by arranging the entries according to the new states. Given the state <sup>J</sup> ð Þ *<sup>L</sup>* , and due to that independency, the matrix *<sup>e</sup>*�*β<sup>C</sup>* � � is given by

$$\mathbb{E}\left(e^{-\beta C}\right) = \mathbb{E}\left(e^{-\beta L}\right)\mathbb{E}\left(e^{-\beta T'}\right). \tag{18}$$

We first introduce two similar LST matrices which differ only with their initial environment: (i) an ð Þ *<sup>n</sup>*<sup>1</sup> � *<sup>n</sup>*<sup>1</sup> matrix ^*<sup>f</sup>* <sup>11</sup>ð Þ 0, *<sup>Q</sup>*, *<sup>β</sup>* whose *ij*th component ^*<sup>f</sup>* <sup>11</sup>ð Þ 0, *<sup>Q</sup>*, *<sup>β</sup>* � � *ij* represents the LST of the time until hitting *Q* in state *j*∈ ℑ1, given *I*ð Þ¼ 0 0,J ð Þ¼ 0 *<sup>i</sup>* <sup>∈</sup> <sup>ℑ</sup><sup>1</sup> and (ii) an ð Þ *<sup>n</sup>*<sup>2</sup> � *<sup>n</sup>*<sup>1</sup> matrix ^*<sup>f</sup>* <sup>21</sup>ð Þ 0, *<sup>Q</sup>*, *<sup>β</sup>* whose *ij*th component ^*<sup>f</sup>* <sup>21</sup>ð Þ 0, *<sup>Q</sup>*, *<sup>β</sup>* � � *ij* represents the LST of the time until hitting *Q* in state *j*∈ ℑ1, given *I*ð Þ¼ 0 0,J ð Þ¼ 0 *i* ∈ ℑ2. Now, define the ð Þ *n* � *n*<sup>1</sup> matrix:

$$
\hat{f}\_{\cdot 1}(\mathbf{0}, \mathbf{Q}, \boldsymbol{\beta}) = \begin{pmatrix} \hat{f}\_{\mathbf{1} 1}(\mathbf{0}, \mathbf{Q}, \boldsymbol{\beta}) \\ \hat{f}\_{\cdot 2 \mathbf{1}}(\mathbf{0}, \mathbf{Q}, \boldsymbol{\beta}) \end{pmatrix}.
$$

Since the ON period ends with environment in ℑ<sup>1</sup> (at time *L*), the ð Þ *n* � *n* matrix *e*�*β<sup>L</sup>* � � has the form:

$$\mathbb{E}(e^{-\beta L}) = \begin{pmatrix} \hat{f}\_{\cdot 1}(\mathbf{0}, Q, \beta) \ \mathbf{0} \end{pmatrix}. \tag{19}$$

Similarly, define the ð Þ *n* � *n*<sup>2</sup> matrix:

$$
\left\langle \hat{f}\_{\cdot 2}(\mathbf{Q}, \mathbf{0}, \boldsymbol{\beta}) \right\rangle = \begin{pmatrix} \left\langle \hat{f}\_{12}(\mathbf{Q}, \mathbf{0}, \boldsymbol{\beta}) \right\rangle \\ \left\langle \hat{f}\_{22}(\mathbf{Q}, \mathbf{0}, \boldsymbol{\beta}) \right\rangle \end{pmatrix}.
$$

(See also (3)). The OFF period ends with environment in ℑ<sup>2</sup> (at time *T*), and thus, the ð Þ *<sup>n</sup>* � *<sup>n</sup>* matrix *<sup>e</sup>*�*β<sup>T</sup>* � �<sup>0</sup> has the form of

$$\mathbb{E}\left(e^{-\beta T'}\right) = \left(\mathbf{0}\_{\mathbf{0}}\hat{f}\_{\cdot \mathbf{2}}(\mathbf{Q}, \mathbf{0}, \boldsymbol{\beta})\right). \tag{20}$$

(Recall that all matrices are given in **Table 1**). Next, we derive the expected discounted costs.

a. **Set up cost**. Let *K*<sup>1</sup> be the setup cost to switch from OFF to ON (at time *T*) and *K*<sup>2</sup> be the setup cost to switch from ON to OFF (at time *L*). Let *SC*ð Þ *β* be the expected discounted set up cost and let **<sup>S</sup>**^ð Þ *<sup>β</sup>* be an ð Þ *<sup>n</sup>* � <sup>1</sup> vector whose *<sup>i</sup>*th component is the expected discounted set up cost given J ð Þ¼ 0 *i* ∈ ℑ,*I*ð Þ¼ 0 0,

$$\hat{\mathbf{S}}(\beta) = E\_i \sum\_{n=1}^{\infty} (\mathbf{K}\_2 \exp\left(-\beta L\_n\right) + \mathbf{K}\_1 \exp\left(-\beta T\_n\right)).\tag{21}$$

Similar technique as before arrives at

$$\begin{split} \text{SC}(\beta) &= \gamma \hat{\mathbf{S}}(\beta) \\ \hat{\mathbf{S}}(\beta) &= \left( \mathbb{I} - \mathbb{E} \left( e^{-\beta \mathbf{C}} \right) \right)^{-1} \left( K\_1 \mathbb{E} \left( e^{-\beta \mathbf{C}} \right) + K\_2 \mathbb{E} \left( e^{-\beta \mathbf{L}} \right) \right) \mathbf{e}. \end{split} \tag{22}$$

Substituting (18) and (19) finalizes the derivation.

b. **Holding cost.** The total expected discounted holding cost *HC*ð Þ¼ *<sup>β</sup> <sup>γ</sup>***H**^ ð Þ *<sup>β</sup>* , where the vector **<sup>H</sup>**^ ð Þ *<sup>β</sup>* is of order ð Þ *<sup>n</sup>* � <sup>1</sup> ; its *<sup>i</sup>*th components is the discounted expected holding cost given J ð Þ¼ 0 *i* ∈ ℑ,*I*ð Þ¼ 0 0*:* Revoking the ergodic theorem for regenerative process, we can write **<sup>H</sup>**^ ð Þ *<sup>β</sup>* in terms of the first cycle and have

$$\hat{\mathbf{H}}(\boldsymbol{\beta}) = \left(\mathbb{I} - \mathbb{E}\left(\boldsymbol{e}^{-\boldsymbol{\beta}\mathbf{C}}\right)\right)^{-1} \Delta\_{\mathbf{h}} \left[\mathbb{E}\left(\int\_{0}^{L} e^{-\boldsymbol{\beta}t} I^{+}(t) dt\right) + \mathbb{E}\left(e^{-\boldsymbol{\beta}\mathbf{L}}\right) \mathbb{E}\left(\int\_{0}^{T} e^{-\boldsymbol{\beta}t} I^{-}(t+L) dt\right)\right].\tag{23}$$

The basic tool we use to derive **E** Ð *L* 0 *e*�*β<sup>t</sup> I* þð Þ*<sup>t</sup> dt* � � and **<sup>E</sup>** *T* Ð 0 0 *e*�*β<sup>t</sup> I* �ð Þ *<sup>t</sup>* <sup>þ</sup> *<sup>L</sup> dt* ! is OST

to the Asmussen–Kella multi-dimensional martingale, as introduced in Lemma 3.2. Here, we consider the process

$$\mathcal{X}(t) = \int\_{v=0}^{t} c\_{\mathcal{I}(v)} dv \; \mathbf{0} \le t \le L,\\ \mathcal{X}(\mathbf{0}) = \mathbf{0}. \tag{24}$$

Chapter *XI*, p. 311 of Asmussen [41] yields that *Ei <sup>e</sup><sup>α</sup>*Xð Þ*<sup>t</sup>* ;<sup>J</sup> ðÞ¼ *<sup>t</sup> <sup>j</sup>* � � <sup>¼</sup> *et*ð Þ *<sup>α</sup>* � � *ij* where ð Þ¼ *α* þ *α* (specifically, þð Þ¼ *α* þ *α*þ). let LðÞ¼� *t* min <sup>0</sup>≤*v*≤*<sup>t</sup>*Xð Þ*v :* The process Lð Þ*t* , known as the *local time,* is a non-decreasing process that increases only whenever *I* þðÞ¼ *t* 0 (for more details on *local time* and its properties, we refer the interested reader to [41]). Next, we let ZðÞ¼ *t* XðÞþ*t* Lð Þ*t :* It is not difficult to see that the latter process up to time *L*, i.e., ð Þ Zð Þ*t* <sup>0</sup> <sup>≤</sup>*t*<*L*, has the same distribution as ð Þ *I t*ð Þ <sup>0</sup>≤*t*<*L:* Finally, define *Y t*ðÞ¼ Lð Þ�*t* ð Þ *β=α t*, for arbitrary *β* ≥0 and *α*< 0, and *W t*ðÞ¼ XðÞþ*t Y t*ðÞ¼ ZðÞ�*t* ð Þ *β=α t:* Since *Y* is adapted and has paths of finite expected total variation on bounded intervals, Theorem 2.1 of Asmussen and Kella [31] leads to the next claim.

*Fluid Inventory Models under Markovian Environment DOI: http://dx.doi.org/10.5772/intechopen.104183*

$$\textbf{Claim 4.1 } The \begin{pmatrix} n \times 1 \end{pmatrix} vector \,\textbf{E} \begin{pmatrix} \int\_0^L e^{-\beta t} I^+(t) dt \end{pmatrix} \text{ is given by:} $$

$$\mathbf{E}\left(\int\_{0}^{L} e^{-\beta t} I^{+}(t)dt\right) = \frac{d}{da} \mathbf{E}\left(\int\_{0}^{L} e^{aZ(t) - \beta t} \mathbf{1}\_{\mathcal{I}(t)} dt\right)\bigg|\_{a=0},\tag{25}$$

where

$$\mathbf{E}\left(\int\_{0}^{L} e^{a\mathcal{Z}(t) - \beta t} \mathbf{1}\_{\mathcal{J}(t)} dt\right) = \left[e^{a\mathcal{Q}} \mathbb{E}\left(e^{-\beta L}\right) - \mathbb{I} - a\left(\mathbf{0}, \hat{\mathbf{L}}(\beta)\right)\right] \left(\mathbb{K}(a) - \beta \mathbb{I}\right)^{-1} \mathbf{e}.\tag{26}$$

**Proof.** The proof and the derivation of **<sup>L</sup>**^ð Þ *<sup>β</sup>* are given in Appendix A. ■ In order to finish the holding cost we have to find the ð Þ *n* � 1 vector **E** *T* Ð 0 0 *e*�*β<sup>t</sup> I* �ð Þ*<sup>t</sup> dt* !*:* Recall that since we are now dealing with the OFF period, for all the matrices in **Table 1**, we have to add the index (�). The OFF period stars at time *L*. We shift the time origin to *<sup>L</sup>* so that the OFF period starts at time 0. Consider <sup>X</sup><sup>~</sup> ð Þ*<sup>t</sup>* similar to (9) but with � and <sup>X</sup><sup>~</sup> ð Þ¼ <sup>0</sup> *<sup>Q</sup>*, <sup>J</sup> ð Þ <sup>0</sup> <sup>∈</sup> <sup>ℑ</sup>*:* Clearly, the latter process up to time *<sup>T</sup>*<sup>0</sup> , i.e., <sup>X</sup><sup>~</sup> ð Þ*<sup>t</sup>* � � <sup>0</sup>≤*t*<*T*0, has the same distribution asð Þ *I t*ð Þ <sup>0</sup> <sup>≤</sup>*t*<*T*<sup>0</sup> *:* Similar arguments to (15)

$$\mathbf{E}\left(\int\_{0}^{T} e^{-\beta t} I(t)dt\right) = \frac{d}{da} \mathbf{E}\left(\int\_{0}^{T} e^{a\bar{\mathcal{X}}(t) - \beta t} dt\right)\bigg|\_{a=0},\tag{27}$$

where

leads to:

$$\mathbf{E}\left(\int\_{0}^{T'} e^{a\vec{\mathcal{X}}(t) - \beta t} dt\right) = \left[\mathbb{E}\left(e^{a\vec{\mathcal{X}}(T') - \beta T'} \mathbf{1}\_{\mathcal{I}(T')}\right) - \mathbb{E}\left(e^{a\vec{\mathcal{X}}(0)} \mathbf{1}\_{\mathcal{I}(0)}\right)\right] \left(\mathbb{K}(a) - \beta\mathbb{I}\right)^{-1} \mathbf{e} \tag{28}$$
 
$$= \left(\mathbb{E}\left(e^{-\beta T'}\right) - e^{-aQ}\mathbb{I}\right) \left(\mathbb{K}(a) - \beta\mathbb{I}\right)^{-1} \mathbf{e}.$$

c. **Production cost.** Let *qi* be the production cost for one unit. We have:

$$\begin{aligned} PC(\beta) &= \gamma \hat{\mathbf{P}}(\beta), \\ \hat{\mathbf{P}}(\beta) &= \left(\mathbb{I} - \mathbb{E}\left(e^{-\beta \mathbf{C}}\right)\right)^{-1} \Delta\_{\mathbf{q}} \mathbf{E}\left(\int\_{0}^{L} p\_{\mathcal{J}(t)} e^{-\beta t} dt\right). \end{aligned} \tag{29}$$

Note that the OFF period is characterized by no production. For the derivation of **E** Ð *L* 0 *<sup>p</sup>*<sup>J</sup> ð Þ*<sup>t</sup> <sup>e</sup>*�*β<sup>t</sup> dt* � � (ON period), let <sup>þ</sup> *<sup>j</sup>* ¼ *diag pi* , *i* ∈ ℑ*<sup>j</sup>* � �, *<sup>j</sup>* <sup>¼</sup> 1, 2 and <sup>þ</sup> <sup>¼</sup> *diag* <sup>þ</sup> <sup>1</sup> , <sup>þ</sup> 2 � �*:* Thus,

$$\mathbb{E}\left(\int\_{0}^{L} p\_{\mathcal{J}(t)} e^{-\beta t} dt\right) = \mathbb{E}\left(\int\_{0}^{L} e^{-\beta t} \mathbf{1}\_{\mathcal{J}(t)} dt\right) \mathbb{P}^{+}\mathbf{e}.\tag{30}$$

Applying (26) with *α* ¼ 0 yields

$$\mathbb{E}\left(\int\_0^L e^{-\beta t} \mathbf{1}\_{\mathcal{I}(t)} dt\right) = \left(\mathbb{E}\left(e^{-\beta L}\right) - \mathbb{I}\right) \left(\mathbb{K}(\mathbf{0}) - \beta\mathbb{I}\right)^{-1},\tag{31}$$

which **completes** the derivation of the production cost.

d. *Lost demand cost.* In our model, backlog is not allowed; any demand which cannot be satisfied immediately is lost. Clearly, there is no unsatisfied demand during the OFF period. During ON period, once level 0 is reached the process stays there until the environment changes to state with a positive growth rate. Assume the process hits 0 at state *i* (for some *c* þ *<sup>i</sup>* < 0Þ, the demand is lost with rate �*c* þ *i* � � until the environmental state changes. Let *wdt* be the cost for a lost unit during a time interval of length *dt* (*w* >0). As a measure for the expected discounted lost demand cost one can use the functional

$$\begin{aligned} \mathbf{U}\mathbf{C}(\boldsymbol{\beta}) &= \gamma \dot{\mathbf{U}}(\boldsymbol{\beta}) \\ \hat{\mathbf{U}}(\boldsymbol{\beta}) &= -w\gamma \left(\mathbb{I} - \mathbb{E}\left(e^{-\boldsymbol{\beta}\mathbf{C}}\right)\right)^{-1} \mathbf{E}\left(\int\_{0}^{L} e^{-\boldsymbol{\beta}t} \boldsymbol{c}\_{\mathcal{I}(t)}^{+} \mathbf{1}\_{\{I^{+}(t) = 0\}} dt\right) \end{aligned} \tag{32}$$

The right term is the expected discounted production loss during the first cycle; it can be written in terms of the local time process.

**Corollary 4.1** *It is easy to verify that*

$$\mathbf{E}\left(\int\_{0}^{L} e^{-\beta t} \mathbf{c}\_{\mathcal{I}(t)}^{+} \mathbf{1}\_{\{I^{+}(t) = 0\}} dt\right) = \mathbf{E}\left(\int\_{0}^{L} e^{-\beta t} d\mathcal{L}(t)\right) = \hat{\mathbf{L}}(\beta)\mathbf{e},\tag{33}$$

*where* **<sup>L</sup>**^ð Þ *<sup>β</sup> is given in* (40)*.*

A simple cost function for the entire system would be the sum, *TC*ð Þ *β* , of these four expected discounted costs:

$$\text{TC}(\beta) = \text{SC}(\beta) + \text{PC}(\beta) + \text{HC}(\beta) + \text{UC}(\beta). \tag{34}$$

### **5. Summary**

During the past few decades, the problem of control of inventory systems has been widely investigated. Many stochastic factors inherent in inventory systems can make it more difficult for managers to plan and control the inventory. Dealing with the randomness of demand, production, and returns, this study considers a continuous-review inventory system where the inventory level is characterized as a fluid process modelled by Markovian environment. The cost structure includes an order cost, a purchase cost, a set up cost, a production cost, an inventory cost, and a lost cost due to unsatisfied demands. By taking a simple probability approach and by applying stopping time

#### *Fluid Inventory Models under Markovian Environment DOI: http://dx.doi.org/10.5772/intechopen.104183*

theory to fluid processes and martingales, the explicit components of the resulting costs are derived. These cost components can be used for optimization purposes. Moreover, the closed-form expression of the components allows us to obtain efficiently and numerically the optimal parameters and enables us to investigate the behavior of the system and to study its properties. From a managerial perspective, our framework can be applied to many industries, in situations where the system is subject to uncertain environment. Our approach appears to be a powerful way to address related inventory problems. The models considered here assume only discrete demand and return sizes. For instance, it seems to be possible to adapt a similar approach in the case of continuous demand size distributions, such as exponential, uniform, or gamma. Another avenue of research would be to extend this framework to include random lead times; in that case, two policies can be applied while shortage, backordering or lost sales. All of these nontrivial extensions are tractable and worthy of study.

## **Appendix A.**

**Proof of claim 4.1.** Theorem 2.1 of Asmussen and Kella [31] yields that the process

$$\begin{split} M(a,t) &= \int\_{v=0}^{t} e^{aW(v)} \mathbf{1}\_{\mathcal{I}(v)} dv \mathbb{K}(a) + e^{aW(0)} \mathbf{1}\_{\mathcal{I}(0)} - e^{aW(t)} \mathbf{1}\_{\mathcal{I}(t)} + a \left( \int\_{v=0}^{t} e^{aW(v)} \mathbf{1}\_{\mathcal{I}(v)} d\mathcal{I}(v) \right) \\ &= \int\_{v=0}^{t} e^{aZ(v)-\beta\mathfrak{e}} \mathbf{1}\_{\mathcal{I}(v)} dv (\mathbb{K}(a) - \beta\mathbb{I}) + e^{a\mathcal{Z}(0)} \mathbf{1}\_{\mathcal{I}(0)} - e^{a\mathcal{Z}(t) - \beta\mathfrak{e}} \mathbf{1}\_{\mathcal{I}(t)} \\ &\quad + a \left( \int\_{v=0}^{t} e^{-\beta\mathfrak{e}} \mathbf{1}\_{\mathcal{I}(v)} d\mathcal{L}(v) \right) \end{split} \tag{35}$$

is an *n*-row vector-valued zero mean martingale. The OST yields ð Þ¼ *M*ð Þ *α*, *L* ð Þ¼ *M*ð Þ *α*, 0 0,

$$\begin{split} & \mathbb{E}\left(\int\_{0}^{L} e^{a\mathcal{Z}(t) - \beta t} \mathbf{1}\_{\mathcal{I}(t)} dt \right) = \left[ \mathbb{E}\left(e^{a\mathcal{Z}(L) - \beta L} \mathbf{1}\_{\mathcal{I}(L)}\right) - \mathbb{E}\left(e^{a\mathcal{Z}(0)} \mathbf{1}\_{\mathcal{I}(0)}\right) \right] \\ & - a\mathbb{E}\left(\int\_{0}^{L} e^{-\beta t} \mathbf{1}\_{\mathcal{I}(t)} d\mathcal{L}(t)\right) \right] (\mathbb{K}(\alpha) - \beta \mathbb{I})^{-1} . \end{split}$$

Since <sup>Z</sup>ð Þ¼ <sup>0</sup> 0 and <sup>J</sup> ð Þ <sup>0</sup> <sup>∈</sup> <sup>ℑ</sup>, we obtain the ð Þ *<sup>n</sup>* � *<sup>n</sup>* matrix *<sup>e</sup><sup>α</sup>*Zð Þ <sup>0</sup> **<sup>1</sup>**<sup>J</sup> ð Þ <sup>0</sup> � � <sup>¼</sup> . Applying Zð Þ¼ *L Q* leads to

$$\mathbb{E}\left(\mathbf{e}^{a\mathcal{Z}(L)-\beta L}\mathbf{1}\_{\mathcal{J}(L)}\right) = \mathbf{e}^{aQ}\mathbb{E}\left(\mathbf{e}^{-\beta L}\right). \tag{36}$$

To finish, we have to derive the ð Þ *<sup>n</sup>* � *<sup>n</sup>* matrix <sup>Ð</sup> *L* 0 *e*�*β<sup>t</sup>* **1**<sup>J</sup> ð Þ*<sup>t</sup> d*Lð Þ*t* � �, which is the expected discounted lost demand until time *L*. Notice that a loss occurs only during

states in ℑ2*:* For that, we introduce the ð Þ *n*<sup>2</sup> � *n*<sup>2</sup> matrix ϒð Þ *β* whose *ij*th component, ϒ*ij*ð Þ *β* , is the expected discounted loss in state *j*∈ ℑ<sup>2</sup> until exiting level 0, given that the process starts at level 0 with state *i* ∈ ℑ2*:*

**Lemma A.1** ϒ*ij*ð Þ *β* satisfies the following system of linear equations:

$$\begin{split} \Upsilon\_{ii}(\boldsymbol{\beta}) &= \frac{-c\_{i}}{\beta} \left( \mathbf{1} + \frac{\mathbb{Q}\_{22}(\boldsymbol{i}, \boldsymbol{i})}{\beta - \mathbb{Q}\_{22}(\boldsymbol{i}, \boldsymbol{i})} \right) + \sum\_{j \neq i} \frac{\mathbb{Q}\_{22}(\boldsymbol{i}, \boldsymbol{j})}{\beta - \mathbb{Q}\_{22}(\boldsymbol{i}, \boldsymbol{i})} \Upsilon\_{ji}(\boldsymbol{\beta}), \\ \Upsilon\_{ij}(\boldsymbol{\beta}) &= \sum\_{k \neq i} \frac{\mathbb{Q}\_{22}(\boldsymbol{i}, \boldsymbol{k})}{\beta - \mathbb{Q}\_{22}(\boldsymbol{i}, \boldsymbol{i})} \Upsilon\_{kj}(\boldsymbol{\beta}). \end{split} \tag{37}$$

**Proof.** Once the inventory process hits 0 in state *i*∈ ℑ2, it stays there for an exponential random time *<sup>ξ</sup><sup>i</sup>* with parameter �22ð Þ *<sup>i</sup>*, *<sup>i</sup> :* With probability 22ð Þ *<sup>i</sup>*, *<sup>j</sup>* �22ð Þ *<sup>i</sup>*, *<sup>i</sup>* the state changes to *j*∈ ℑ, and the expected discounted loss from state *j* is ϒ *<sup>j</sup>*ð Þ*i :* By conditioning on the first state visited after *i*, we readily obtain that

ϒ*ii*ð Þ¼� *β E* ð *ξi t*¼0 *cie* �*βt dt* 0 @ 1 A þ *Ei e* �*βξ<sup>i</sup>* � � X *i*6¼*j j* ∈ ℑ<sup>2</sup> 22ð Þ *i*, *j* �22ð Þ *i*, *i* ϒ*ji*ð Þ *β* , ϒ*ij*ð Þ¼ *β Ei e* �*βξ<sup>i</sup>* � � X *k*6¼*i k*∈ ℑ<sup>2</sup> 22ð Þ *i*, *k* �22ð Þ *i*, *i* ϒ*kj*ð Þ *β :* (38)

Solving (38) with respect to ϒ*ii*ð Þ *β* and ϒ*ij*ð Þ *β* returns (37) (note that *E* Ð *ξi t*¼0 *cie*�*β<sup>t</sup> dt* ! <sup>¼</sup> *ci <sup>β</sup>* <sup>1</sup> <sup>þ</sup> 22ð Þ *<sup>i</sup>*, *<sup>i</sup> β*�22ð Þ *i*, *i* � � and *Ei <sup>e</sup>*�*βξ<sup>i</sup>* � � <sup>¼</sup> �22ð Þ *<sup>i</sup>*, *<sup>i</sup> <sup>β</sup>*�22ð Þ *<sup>i</sup>*, *<sup>i</sup>* <sup>Þ</sup>). ■.

Now, we apply ϒð Þ *β* to the derivation of the loss until time *L:* Since the lost demand occurs only for states in ℑ2, the matrix Ð *L v*¼0 *<sup>e</sup>*�*β<sup>v</sup>***1**<sup>J</sup> ð Þ*<sup>v</sup> <sup>d</sup>*Lð Þ*<sup>v</sup>* � � has the form

$$\mathbb{E}\left(\int\_{0}^{L} e^{-\beta t} \mathbf{1}\_{\mathcal{I}(t)} d\mathcal{L}(t)\right) = \left(\mathbf{0}, \mathbf{\hat{L}}(\beta)\right) \tag{39}$$

where **<sup>L</sup>**^ð Þ *<sup>β</sup>* is an ð Þ *<sup>n</sup>* � *<sup>n</sup>*<sup>2</sup> matrix whose *ij*th component is the expected discounted loss in state *<sup>j</sup>*<sup>∈</sup> <sup>ℑ</sup><sup>2</sup> until *<sup>L</sup>*, given <sup>J</sup> ð Þ¼ <sup>0</sup> *<sup>i</sup>*<sup>∈</sup> <sup>ℑ</sup>. Let **<sup>L</sup>**^*i*ð Þ *<sup>β</sup>* , *<sup>i</sup>* <sup>¼</sup> 1, 2 be an ð Þ *ni* � *<sup>n</sup>*<sup>2</sup> submatrix of **<sup>L</sup>**^ð Þ *<sup>β</sup>* includes all rows corresponding to states in <sup>ℑ</sup>*i*, *<sup>i</sup>* <sup>¼</sup> 1, 2, such that

$$
\hat{\mathbf{L}}(\boldsymbol{\beta}) = \begin{pmatrix} \hat{\mathbf{L}}\_1(\boldsymbol{\beta}) \\ \hat{\mathbf{L}}\_2(\boldsymbol{\beta}) \end{pmatrix}. \tag{40}
$$

It is easy to verify that

$$\begin{split} \hat{\mathbf{L}}\_{1}(\boldsymbol{\beta}) &= {}^{Q}\boldsymbol{\Psi}(\boldsymbol{\beta}) \Big( \mathbf{Y}(\boldsymbol{\beta}) + (\boldsymbol{\beta}\mathbb{I} - \mathbb{Q}\_{22})^{-1} \mathbb{Q}\_{21} \hat{\mathbf{L}}\_{1}(\boldsymbol{\beta}) \Big), \\ \hat{\mathbf{L}}\_{2}(\boldsymbol{\beta}) &= {}^{Y}(\boldsymbol{\beta}) + (\boldsymbol{\beta}\mathbb{I} - \mathbb{Q}\_{22})^{-1} \mathbb{Q}\_{21} \hat{\mathbf{L}}\_{1}(\boldsymbol{\beta}). \end{split} \tag{41}$$

*Fluid Inventory Models under Markovian Environment DOI: http://dx.doi.org/10.5772/intechopen.104183*

Lost demand occurs when the process drops to level 0 avoiding *Q*, with LST *<sup>Q</sup>* <sup>Ψ</sup>ð Þ *<sup>β</sup> :* From that point, <sup>ϒ</sup>ð Þ *<sup>β</sup>* is the discounted lost demand. The term *<sup>β</sup>* � <sup>22</sup> ð Þ�<sup>1</sup> <sup>21</sup> is the discounted time until exiting level 0 with environmental state in ℑ<sup>1</sup> and starting again with LST **<sup>L</sup>**^1ð Þ *<sup>β</sup> :* The matrix **<sup>L</sup>**^2ð Þ *<sup>β</sup>* is derived similarly.
