**4. A fluid EPQ-type inventory model**

Similar as above, let *I t*ð Þ be the inventory level at a time *t:* The process *I t*ð Þ can be partitioned into two parts, *I* þð Þ*t* and *I* �ð Þ*t :* The first part of the cycle, *I* þð Þ*t* , is the ON period and this period ends whenever *I* þð Þ*t* reaches a predetermined level *Q*. The second part of the cycle, *I* �ð Þ*t* , is the OFF period and this is the time until *I* �ð Þ*t* drops to level 0. The ON period is characterized by stochastic inputs, production and returns, and outputs, demand. However, no production occurs during the OFF period, and thus, the OFF period is characterized by stochastic returns and demand. Here, the analysis is more challenging due to the periods ON and OFF. Denote by *p*1, … , *pn* � �⊂ð Þ 0, <sup>∞</sup> the production rates, by f g *<sup>d</sup>*1, … , *dn* <sup>⊂</sup>ð Þ 0, <sup>∞</sup> the demand rates, and by f g *r*1, … ,*rn* ⊂ð Þ 0, ∞ the returns rates. The rate at which the inventory is filled at time *t* is determined by the current environmental state J ð Þ*t* . During the ON period (OFF period) and as long as J ðÞ¼ *t i*, the growth rate is the difference of the production and return rates (only the return rate) and the demand rate, *c* þ *<sup>i</sup>* ¼ *ri* þ *pi* � *di* (*c*� *<sup>i</sup>* ¼ *ri* � *di*); similarly as before, *ci*, *i* ¼ 1, *::*, *n* may be either negative or positive. We do not allow backlog; thus when *I* þð Þ*t* drops to level 0 (due to high demands), it stays there as long as the environmental growth rate is negative and until the environmental state changes to some positive growth rate. Note that the behavior of the process during each period is different. During the OFF period, we enforce that P*<sup>n</sup> <sup>i</sup>*¼<sup>1</sup>*πic*� *<sup>i</sup>* < 0 (a necessary and sufficient condition for the stability of the OFF process). During the ON period, we assume that P*<sup>n</sup> <sup>i</sup>*¼<sup>1</sup>*πic* þ *<sup>i</sup>* >0 (although with the absence of this, *I* þð Þ*t* is stable due to the

#### **Figure 2.**

*A typical path of the fluid EPQ-type inventory process.*

reflection at level 0, however, it is a realistic assumption in order to avoid high lost demand).

A typical sample path of the inventory process is given in **Figure 2**. As we see, the inventory process is a semi-regenerative process which alternates between ON periods and OFF periods. We further note that if the ON periods are deleted from the sample path and the OFF periods are glued together we obtain a *fluid EOQ-type* model with refilling every time level 0 is reached. This holds with one exception; while each cycle under the *fluid EOQ-type* process starts with a descending state, it's not necessarily holds in the *fluid EPQ-type* model (a detailed explanation is given below).

Define the following stopping times:

$$\begin{aligned} T\_0 &= \mathbf{0} \\ T\_k &= \inf \left\{ t > L\_k : I(t) = \mathbf{0} \right\} \ k = \mathbf{1}, 2 \dots \\ L\_k &= \inf \left\{ t \ge T\_{k-1} : I(t) = \mathbf{Q} \right\} k = \mathbf{1}, 2 \dots \end{aligned}$$

*Tk*, *k* ¼ 1, 2, … are the times of switchings from OFF to ON, and *Ln*, *n* ¼ 1, 2, … are the time instants of switchings from ON to OFF. Thus, *I t*ð Þ is a semi-regenerative process with *T*<sup>0</sup> *ns* are semi-regenerative points. Define the *k*th cycle as the time elapsed between *Tk*�<sup>1</sup> and *Tk*, *k* ¼ 1, 2, *::* and let *Ck* ¼ *Tk* � *Tk*�1, *k* ¼ 1, 2 … be the *k*th cycle length. We use the generic form *L* ¼ *L*1, *T* ¼ *T*1, *T*<sup>0</sup> ¼ *T* � *L* and *C* ¼ *C*<sup>1</sup> (so *C* ¼ *L* þ *T*<sup>0</sup> ). Note that conditioning on the state at time *L* and the common background environmental process, the two process *I* þð Þ*t* <sup>0</sup> <sup>≤</sup>*t*≤*<sup>L</sup>* (the ON period) and *I* �ð Þ*t <sup>L</sup>*≤*t*≤*<sup>T</sup>* (the OFF period) are independent.

We construct diagonal matrices <sup>þ</sup> *<sup>j</sup>* ¼ *diag*f *c* þ *i* , *i*∈ ℑ*j*g, � *<sup>j</sup>* ¼ *diag*f *c*� *i* , *i* ∈ ℑ*j*g, *j* ¼ 1, 2 and <sup>þ</sup> ¼ *diag* <sup>þ</sup> <sup>1</sup> , <sup>þ</sup> 2 and � <sup>¼</sup> *diag* � <sup>1</sup> , � 2 from these rates. Regarding the fluid EPQ-type model, each ON/OFF period has one type of rates <sup>þ</sup> or �Þ; hence, given the state at switching epoch and the common environment, the ON/OFF periods are independent. Now, we can analyze the inventory level within each period independently using MMFF process. Specifically, we consider *I* <sup>þ</sup>, <sup>þ</sup> and Ψþð Þ *β* (*I* �, �, Ψ�ð Þ *β* ) corresponding to the ON (OFF) period. Note that, for this model, each LST matrix should be derived for ON (marked as +) and OFF (marked as �) processes. However, we do not insert the marks + or � corresponding to ON/OFF processes; it should be clear from the context which of the marks applies.

### **4.1 The cost functionals**

We consider four costs: (a) the setup cost, (b) the holding cost of the inventory, (c) the production cost, and (d) the unsatisfied demand cost. For the derivation of the costs, we first need to derive the expected discounted cycle length ð Þ *n* � *n* matrix *e*�*β<sup>C</sup>* � �.
