**Table 1.**

*Transform matrices.*


**Table 2.** *LST of first passage times.*

#### **2.2 The multi-dimensional martingale**

Let f g Xð Þ*t* , *t* ≥0 be a right continuous Markov modulated Lévy process with modulating process f g J ð Þ*t* , *t* ≥0 which is a right continuous irreducible finite state space continuous time Markov chain. Let f g *Y t*ð Þ, *t* ≥0 be an adapted continuous process with a finite expected variation on finite intervals and let ZðÞ¼ *t* XðÞþ*t Y t*ð Þ*:* Asmussen and Kella [31] have shown, that for such a process, the matrix with elements *Ei <sup>e</sup>α*Xð Þ*<sup>t</sup>* ; <sup>J</sup> ðÞ¼ *<sup>t</sup> <sup>j</sup>* � � has the form of *<sup>e</sup><sup>t</sup>*ð Þ *<sup>α</sup>* for some matrix ð Þ *<sup>α</sup> :* Theorem 2.1 in Asmussen and Kella [31] yields that under certain conditions on f g Zð Þ*t* , *t*≥0 , the multi-dimensional process

$$M(a,t) = \int\_{v=0}^{t} \epsilon^{a\mathcal{Z}(v)} \mathbf{1}\_{\mathcal{J}(v)} dv \mathbb{K}(a) + \epsilon^{a\mathcal{Z}(0)} \mathbf{1}\_{\mathcal{J}(0)} - \epsilon^{a\mathcal{Z}(t)} \mathbf{1}\_{\mathcal{J}(t)} + a \int\_{v=0}^{t} \epsilon^{a\mathcal{Z}(v)} \mathbf{1}\_{\mathcal{J}(v)} d\mathcal{Y}(v) \tag{1}$$

is a (row) vector-valued zero mean martingale. Some of the relevant functionals in this paper will be obtained by applying the OST (or Doob's optional sampling theorem, see Doob [39]) to appropriate special cases of For our models, the inventory level ℐ is a special case of Xð Þ*t* and has piecewise linear sample paths, with slope *cj* on intervals where J ðÞ¼ *t j:*

The outline of the chapter is as follows. We start with the ordering model and introduce an extension to the EOQ model, a *fluid EOQ-type* inventory model. Then, using the *fluid EOQ-type* model as a key, our study is generalized to include production facility, the *fluid EPQ-type* model.

### **3. A fluid EOQ-type inventory model**

The simple *Economic Ordering Quantity* (EOQ) model assumes an inventory model of infinite horizon for a single item. The demand occurs at a constant rate and each time the inventory level hits a level 0, a fixed and immediate order of size *Q* is places. *Fluid Inventory Models under Markovian Environment DOI: http://dx.doi.org/10.5772/intechopen.104183*

EOQ considers the timing of reordering, the cost incurred to place an order, and the costs to store merchandise. Here, we generalize the traditional EOQ model and consider demand and return rates that depend on the background environment, and varying state-dependent holding and ordering costs. We start with the mathematical description of the model.

Let ℐ ¼ f g *I t*ð Þ : *t*≥0 be the on-hand inventory level at time *t*. The rate of change of the level is modulated by a continuous time Markov chain (CTMC) f g J ð Þ*t* : *t*≥ 0 on a finite state space <sup>ℑ</sup> <sup>¼</sup> f g 1, 2, … , *<sup>n</sup>* with a generator matrix <sup>¼</sup> *Qij* h i. As long as J ð Þ*t* is in state *i*, there is a demand at rate *di* and a return at rate *ri:* The net rate is thus *ci* ¼ *ri* � *di:* Note that *ci* may be either negative or positive. Accordingly, we have two disjoint sets ð Þ ℑ1, ℑ<sup>2</sup> , ℑ ¼ ℑ<sup>1</sup> ∪ ℑ2, where ℑ<sup>1</sup> is a non-empty set of increasing rates ℑ<sup>1</sup> ¼ f g *i*∈ ℑ : *ci* >0 and ℑ<sup>2</sup> is a non-empty set of the decreasing rates ℑ<sup>2</sup> ¼ f g *i*∈ ℑ : *ci* <0 *:* Let j j ℑ<sup>1</sup> ¼ *n*<sup>1</sup> and j j ℑ<sup>2</sup> ¼ *n*2; thus *n*<sup>1</sup> þ *n*<sup>2</sup> ¼ *n*. Let *π* ¼ ½ � *π*1, … , *π<sup>n</sup>* be the limiting distribution of the J ð Þ*t* process, i.e. *π* is the unique solution to

$$
\boldsymbol{\pi} \mathbb{Q} = 0, \boldsymbol{\pi} \mathbf{e} = 1.
$$

The system is stable if and only if the expected input rate is negative, i.e.,

$$\sum\_{i=1}^{n} \pi\_i c\_i < 0.$$

When *I t*ð Þ down-crosses level 0, an order of size *Q* >0 from an external supplier is placed which arrives instantaneously. Thus, immediately after the down-crossing to emptiness by the inventory level in state *i*, the process restarts at level *Q:* It should be noted, that the EOQ model is a special case of the base–stock polices, in particular, the ð Þ *s*, *S* -type, where the reorder level *s* ¼ 0, the replenishment up-to-level *S* ¼ *Q* and zero lead time.

Let *Tk* be the time of the *k*th jump ð Þ *T*<sup>0</sup> ¼ 0 and J *<sup>k</sup>* ¼ J ð Þ *Tk* be the environmental state at *Tk* (just after the jump). Note that since an order is placed when the inventory drops to level 0, the state J *<sup>k</sup>* has to be a descending one, i.e., J *<sup>k</sup>* ∈ ℑ2*:* We assume that over *Tk*�<sup>1</sup> ½ Þ , *Tk k* ¼ 1, 2*::*, the process J ð Þ*t* , *t*∈*Tk*�<sup>1</sup> f , *Tk*Þg is an irreducible CTMC (continuous time Markov chain) on ℑ. We call the points where the process jumps up (the replenishment times) *order points*. They form a semi-renewal process where *T*<sup>0</sup> *ks* are the semi regenerative points of the process (see Ross [40]). Define the *k*th cycle as the time elapsed between *Tk*�<sup>1</sup> and *Tk*, *k* ¼ 1, 2, *::*. Denote by *Ck* ¼ *Tk* � *Tk*�1, *k* ¼ 1, 2 … ð Þ *C*<sup>0</sup> ¼ 0 the inter-replenishment times; let *C* ¼ *C*1. **Figure 1** illustrates a sample path of the process *I t*ð Þ, *t* ≥0; for simplicity, we assume that *I*ð Þ¼ 0 *Q*, J ð Þ 0 ∈ ℑ*<sup>n</sup>* and let *γ* ¼ *γ*1, *γ*2, … , *γ<sup>n</sup>*<sup>2</sup> � � be the initial probability vector of <sup>J</sup> ð Þ <sup>0</sup> .

#### **3.1 The cost structure**

Our cost structure composed of two components: (a) an order cost, including a fixed cost whenever an order is placed and a purchasing cost; (b) a holding cost for the stock. We further assume that the cost rates are determined by the environment at the *order points;* in real-world, order rates usually depend on the state of the economy, the season, and change accordingly. Specifically, if the state at an *order point* is J *<sup>k</sup>* ¼ *i* ∈ ℑ2, the fixed ordering cost is *Ki* for an order (typically cost of ordering and shipping and handling), the cost to purchase one item from an external supplier is *qi* ,

#### **Figure 1.**

*A typical path of the fluid EOQ-type inventory process.*

and the cost to hold one item in inventory during a time interval of length *dt* is *hidt* during all the cycle *k* in which J *<sup>k</sup>* ¼ *i*∈ ℑ<sup>2</sup> (meaning that the holding cost for one unit is constant between two consecutive order replacements, i.e. *h t*ðÞ¼ *hi* for *t*∈ *Tk*, *Tk*þ<sup>1</sup>Þ and J *<sup>k</sup>* ¼ *i* ∈ ℑ2). We next introduce the functionals indicating the expected discounted costs using a discount factor *β* > 0*:*

a. *Order cost*. If J *<sup>k</sup>* ¼ *i*∈ ℑ2, an order of size *Q* is immediately replenished up to level *I T*ð Þ¼ *<sup>k</sup> Q*. The order cost is *Ki* þ *qi Q:* Let *OC*ð Þ *β* be the expected discounted order cost and let **<sup>O</sup>**^ ð Þ *<sup>β</sup>* be an ð Þ *<sup>n</sup>*<sup>2</sup> � <sup>1</sup> vector whose *<sup>i</sup>*th component *<sup>O</sup>*^*i*ð Þ *<sup>β</sup>* is given by

$$\hat{O}\_i(\beta) = E\_i \left[ \sum\_{k=0}^{\infty} e^{-\beta T\_k} \left( K\_i + q\_i Q \right) \right],\tag{2}$$

i.e., the *expected* discounted order cost, given J ð Þ¼ 0 *i* ∈ ℑ2,*I*ð Þ¼ 0 *Q:* Then we have *OC*ð Þ¼ *<sup>β</sup> <sup>γ</sup>* � **<sup>O</sup>**^ ð Þ *<sup>β</sup> :* Applying **Table 2**, let <sup>0</sup> ^*<sup>f</sup>* <sup>22</sup>ð Þ *<sup>Q</sup>*, 0, *<sup>β</sup>* be an ð Þ *<sup>n</sup>*<sup>2</sup> � *<sup>n</sup>*<sup>2</sup> matrix whose *ij*th components is

$$\left(\hat{\varnothing}^{\hat{I}}\_{22}(Q,\mathbf{0},\beta)\right)\_{\vec{\eta}} = e^{\mathbb{E}(\beta)Q}, \quad i \in \mathfrak{F}\_2, j \in \mathfrak{F}\_2. \tag{3}$$

The *component* <sup>0</sup> ^*<sup>f</sup>* <sup>22</sup>ð Þ *<sup>Q</sup>*, 0, *<sup>β</sup>* � � *ij* represents the LST of the time until the process hits level 0 in state *j*∈ ℑ2, given J ð Þ¼ 0 *i* ∈ ℑ2,*I*ð Þ¼ 0 *Q* (for a proof, see Ramaswami [33], Theorem 5).

**Lemma 3.1** *The total expected discounted order cost vector* **<sup>O</sup>**^ ð Þ *<sup>β</sup> of order n*ð Þ <sup>2</sup> � <sup>1</sup> *satisfies the following equation:*

$$
\hat{\mathbf{O}}(\boldsymbol{\beta}) = \left(\mathbb{I} - \hat{\boldsymbol{\lg}}\_{\mathfrak{Q}}(\boldsymbol{Q}, \mathbf{0}, \boldsymbol{\beta})\right)^{-1} \boldsymbol{\Delta}\_{\mathbf{K} + \mathbf{q}\boldsymbol{Q}} \mathbf{e}.\tag{4}
$$

where Δ**<sup>K</sup>**þ**q***<sup>Q</sup>* ¼ *diag Ki* þ *qi Q* � �, *i* ∈ <sup>ℑ</sup><sup>2</sup> *:*

**Proof.** It is easy to verify that **<sup>O</sup>**^ ð Þ *<sup>β</sup>* can be written as (recall that *<sup>C</sup>* is the time of the next order):

$$
\hat{\mathbf{O}}(\beta) = \Delta\_{\mathbf{K} + \mathbf{q}Q} \mathbf{e} + \mathbb{E}(e^{-\beta C}) \hat{\mathbf{O}}(\beta), \tag{5}
$$

where the ð Þ *<sup>n</sup>*<sup>2</sup> � *<sup>n</sup>*<sup>2</sup> matrix *<sup>e</sup>*�*β<sup>C</sup>* � � is the LST of the cycle length; its *ij*th component is given by

$$\mathbb{E}\left(e^{-\beta C}\right)\_{\vec{\eta}} = \mathbb{E}\left(e^{-\beta C}\mathbf{1}\_{\{\text{level 0} \text{ hit at time } C \text{ in phase } j\}} \, \middle|\, \mathcal{T}(\mathbf{0}) = i, I(\mathbf{0}) = Q\right). \tag{6}$$

Applying the fluid model *<sup>e</sup>*�*β<sup>C</sup>* � � <sup>¼</sup> <sup>0</sup> ^*<sup>f</sup>* <sup>22</sup>ð Þ *<sup>Q</sup>*, 0, *<sup>β</sup>* and solving (5) for **<sup>O</sup>**^ ð Þ *<sup>β</sup>* we obtain (4) ■

a. *Holding cost.* The expected discounted holding cost can be expressed as

$$HC(\beta) = E\left(\bigcap\_{0}^{\infty} h(t)e^{-\beta t}I(t)dt\right). \tag{7}$$

Let **<sup>H</sup>**^ ð Þ *<sup>β</sup>* be an ð Þ *<sup>n</sup>*<sup>2</sup> � <sup>1</sup> vector whose *<sup>i</sup>*th component is given by

$$\hat{H}\_i(\beta) = E\_i \left( \bigcap\_{0}^{\infty} h(t)e^{-\beta t} I(t)dt \right) \tag{8}$$

i.e., the expected discounted holding cost, given J ð Þ¼ 0 *i* ∈ ℑ2*:* Thus, we have *HC*ð Þ¼ *<sup>β</sup> <sup>γ</sup>* � **<sup>H</sup>**^ ð Þ *<sup>β</sup> :*

**Lemma 3.2** *The vector* **<sup>H</sup>**^ ð Þ *<sup>β</sup> of order n*ð Þ <sup>2</sup> � <sup>1</sup> *satisfies the following equation:*

$$
\hat{\mathbf{H}}(\boldsymbol{\beta}) = \left(\mathbb{I} - \,\_0\hat{\mathbf{f}}\_{22}(\mathbf{Q}, \mathbf{0}, \boldsymbol{\beta})\right)^{-1} \Delta\_{\mathbf{h}} \mathbf{E} \left(\int\_{t=0}^{\mathbf{C}} e^{-\beta t} I(t) dt\right).
$$

**Proof.** Applying the regenerative theory, and similar to Lemma 3.1, the vector **<sup>H</sup>**^ ð Þ *<sup>β</sup>* can be written as

$$\hat{\mathbf{H}}(\beta) = \Delta\_{\mathbf{h}} \mathbf{E} \left( \int\_{t=0}^{C} e^{-\beta t} I(t) dt \right) + \mathbb{E} \left( e^{-\beta C} \right) \hat{\mathbf{H}}(\beta)$$

(The first vector **E** Ð *<sup>C</sup> <sup>t</sup>*¼<sup>0</sup>*e*�*β<sup>t</sup> I t*ð Þ*dt* � � is the expected discounted inventory level of the first cycle). Now, we use the OST to the multi-dimensional martingale to find the ð Þ *<sup>n</sup>*<sup>2</sup> � <sup>1</sup> vector **<sup>E</sup>** <sup>Ð</sup> *<sup>C</sup> <sup>t</sup>*¼<sup>0</sup>*e*�*β<sup>t</sup> I t*ð Þ*dt* � �*:* For <sup>J</sup> ð Þ¼ <sup>0</sup> *<sup>i</sup>* <sup>∈</sup> <sup>ℑ</sup>2,*I*ð Þ¼ <sup>0</sup> *<sup>Q</sup>*, consider a Lévy process f g X*i*ð Þ*t* as follows:

$$\mathcal{X}\_i(t) = \mathcal{X}\_i(\mathbf{0}) - \int\_{v=0}^t c\_{\mathcal{I}(v)} dv \quad \mathbf{0} \le t < \mathbf{C}, \quad \mathcal{X}\_i(\mathbf{0}) = -\mathbf{Q}. \tag{9}$$

It is not difficult to see that the latter process up to time *C*, i.e., ð Þ X*i*ð Þ*t* <sup>0</sup> <sup>≤</sup>*t*<*C*, has the same distribution as ð Þ �*I t*ð Þ <sup>0</sup> <sup>≤</sup>*t*<*C:* Chapter XI, p. 311 of Asmussen [41] yields that

$$E\_i\left[e^{a\mathcal{X}\_i(t)};\mathcal{J}(t)=j\right] = \left(e^{t\mathbb{K}(a)}\right)\_{ij}$$

where

$$\mathbb{K}(a) = \mathbb{Q} - a\mathbb{C}.\tag{10}$$

Let *Y t*ðÞ¼�ð Þ *β=α t* (for an arbitrary *α*> 0Þ and let Z*i*ðÞ¼ *t* X*i*ðÞþ*t Y t*ð Þ*:* Since *Y t*ð Þ is adapted and has paths of a finite expected variation, the process

$$M\_i(a,t) = \int\_{v=0}^t \epsilon^{aZ\_i(v)} \mathbf{1}\_{\mathcal{I}(v)} dv \mathbb{K}(a) + \epsilon^{aZ\_i(0)} \mathbf{1}\_{\mathcal{I}(0)} - \epsilon^{aZ\_i(t)} \mathbf{1}\_{\mathcal{I}(t)} + a \int\_{v=0}^t \epsilon^{aZ\_i(v)} \mathbf{1}\_{\mathcal{I}(v)} d\mathcal{Y}(v)$$

$$= \int\_{v=0}^t \epsilon^{a\mathcal{X}\_i(v) - \beta v} \mathbf{1}\_{\mathcal{I}(v)} d\nu(\mathbb{K}(a) - \beta \mathbb{I}) + \epsilon^{a\mathcal{X}\_i(0)} \mathbf{1}\_{\mathcal{I}(0)} - \epsilon^{a\mathcal{X}\_i(t) - \beta t} \mathbf{1}\_{\mathcal{I}(t)} \tag{11}$$

is an *n*2-dimensional row vector-valued zero mean martingale. The OST yields *EMi*ð Þ¼ *α*, 0 *EMi*ð Þ¼ *α*,*C* 0, i.e.,

$$E\_i\left(\int\_{t=0}^{C} e^{a\mathbf{X}\_i(t) - \beta t} dt\right) = \left[\mathbb{E}\_i\left(e^{a\mathbf{X}\_i(C) - \beta C} \mathbf{1}\_{\mathcal{I}(C)}\right) - \mathbb{E}\_i\left(e^{a\mathbf{X}\_i(0)} \mathbf{1}\_{\mathcal{I}(0)}\right)\right] \left(\mathbb{K}(a) - \beta \mathbb{I}\right)^{-1} \mathbf{e}.\tag{12}$$

Obviously, we have *Ei <sup>e</sup>α*X*i*ð Þ <sup>0</sup> **<sup>1</sup>**<sup>J</sup> ð Þ <sup>0</sup> � � <sup>¼</sup> *<sup>e</sup>*�*α<sup>Q</sup>* **for** *<sup>i</sup>* <sup>∈</sup> <sup>ℑ</sup>2, or in an ð Þ *<sup>n</sup>*<sup>2</sup> � *<sup>n</sup>* matrix form

$$\mathbb{E}\left(e^{a\mathcal{X}(0)}\mathbf{1}\_{\mathcal{I}(0)}\right) = \begin{pmatrix} \mathbf{0} \ e^{-aQ}\mathbb{I} \end{pmatrix}.\tag{13}$$

(Here, the ð Þ *n*<sup>2</sup> � *n*<sup>1</sup> zero matrix arising due to the fact that J ð Þ 0 ∈ ℑ2). Next, we have to derive **<sup>E</sup>***<sup>i</sup> <sup>e</sup><sup>α</sup>Xi*ð Þ� *<sup>C</sup> <sup>β</sup><sup>C</sup>***1**<sup>J</sup> ð Þ *<sup>C</sup>* � �*:* Since *Xi*ð Þ¼ *<sup>C</sup>* 0, the fluid method yields the ð Þ *n*<sup>2</sup> � *n* matrix form

$$\mathbb{E}(e^{-\beta \mathbf{C}} \mathbf{1}\_{\mathcal{I}(C)}) = \left(\mathbf{0}\_{\mathbf{0}} \hat{f}\_{22}(\mathbf{Q}, \mathbf{0}, \boldsymbol{\beta})\right). \tag{14}$$

Substituting (13) and (14) in (12) we obtain the ð Þ *n*<sup>2</sup> � 1 vector

$$\mathbf{E}\left(\int\_{t=0}^{C} e^{a\mathcal{X}(t) - \beta t} dt\right) = \left(\left(\mathbf{0}\_{\emptyset}\hat{f}\_{22}(Q, \mathbf{0}, \beta)\right) - \left(\mathbf{0}\_{\bullet}e^{-aQ}\mathbb{I}\right)\right) \left(\mathbb{K}(a) - \beta\mathbb{I}\right)^{-1}\mathbf{e}.\tag{15}$$

Finally, take the derivative of both sides of (15) with respect to *α*, let *α* ¼ 0 and note that

*Fluid Inventory Models under Markovian Environment DOI: http://dx.doi.org/10.5772/intechopen.104183*

$$\mathbf{E}\left(\int\_{t=0}^{C} e^{-\beta t} \mathcal{X}(t) dt\right) = -\mathbf{E}\left(\int\_{t=0}^{C} e^{-\beta t} I(t) dt\right).$$

leads to

$$\mathbf{E}\left(\int\_{a=0}^{c} e^{-\beta t} I(t)dt\right) = -\frac{d}{da}\left[\left(\mathbf{0}\_{\emptyset}\hat{f}\_{22}(Q,\mathbf{0},\beta)\right) - \left(\mathbf{0}\_{\,,}e^{-aQ}\mathbb{I}\right)\left(\mathbb{K}(a) - \beta\mathbb{I}\right)^{-1}\right]\mathbf{e}\Big|\_{a=0}.\tag{16}$$

Accordingly, the expected discounted total cost is

$$TC(\beta) = OC(\beta) + HC(\beta). \tag{17}$$

■ **Remark 1** *It is easy to extend this model to include an order size determined by the environment at the order point, i.e., if* J ð Þ¼ *Tk i*∈ ℑ2*, an order of Qi items is placed (and, accordingly, I*ð Þ¼ 0 *Qi with probability γi*, *i*∈ ℑ2*). For that we define the vector* **Q** ¼ *Q*1, … , *Qn*<sup>2</sup> � �*: Now, the entries* <sup>0</sup> ^*<sup>f</sup>* <sup>22</sup>ð Þ *<sup>Q</sup>*, 0, *<sup>β</sup>* � � *ij are obtained by replacing Q with Qi in* (3)*. The order cost is given by* (4) *with* Δ**K**þ**qQ** *replacing* Δ**K**þ**q***<sup>Q</sup>* , *and the holding cost is given by substituting* <sup>Δ</sup>*<sup>e</sup>*�*α***<sup>Q</sup>** *instead e*�*α<sup>Q</sup> in* (13) *(here,* <sup>X</sup>*i*ð Þ¼� <sup>0</sup> *Qi).*

Using the approach for the *fluid EOQ-type* model as a key, we next include a production facility and introduce the *fluid EPQ-type* model.
