**Chapter 8** Fluid Inventory Models under Markovian Environment

*Yonit Barron*

### **Abstract**

Today's products are subject to fast changes due to market conditions, short life cycles, and technological advances. Thus, an important problem in inventory planning is how to effectively manage the inventory control in a dynamic and stochastic environment. The traditional Economic Order Quantity (EOQ) and Economic Production Quantity (EPQ) both are widely and successfully used models of inventory management. However, both models assume constant and fixed parameters over time. Unfortunately, most of these assumptions are unrealistic. In this study, we generalize the EOQ and EPQ models and study production-inventory fluid models operating in a stochastic environment. The inventory level increases or decreases according to a fluid-flow rate modulated by an *n*-state continuous time Markov chain (CTMC). Our main objective is to minimize the expected discounted total cost which includes ordering, purchasing, production, set up, holding, and shortage costs. Applying regenerative theory, optional sampling theorem (OST) to the multi-dimensional martingale and fluid flow techniques, we develop methods to obtain explicit formulas for these cost functionals. As such, we provide managers with a useful framework and an efficient and easy-to-implement tool to coop with different demand–supply patterns.

**Keywords:** Inventory/production, Markov chain, Fluid flow, Renewal theory, Martingales, EOQ, EPQ

### **1. Introduction**

An important problem in inventory planning is how to effectively manage the inventory control in a dynamic and stochastic environment. Particularly, today's products are subject to fast changes due to market conditions, short life cycles, and technological advances. The uncertainties in the supply-chain hierarchy make inventory control very challenging and the structure of the optimal policy is still unknown; thus, both researchers and practitioners are focusing on relatively simple control policies. This focus is especially true in practice where there are uncertainties in the production rate, demand rate, and lead time. The specification of the production rates, order quantities, the storage capacity and the backlog possibilities has to take into account the costs for ordering, production, the holding of inventory, backlogging, and lost sales. For example, for a factory Manufacturer, fixing the production rate at some high level or ordering a large quantity avoids backlogs but may cause high production/ order cost and inventory costs; however, fixing the production rate at some low level, or ordering many but small quantities may lead to severe backlogging costs and the loss of sale opportunities. An inventory control policy governs inventory replenishment decisions by specifying when and how many items should be ordered or produced, thus, the planning of such a supply chain subject to market uncertainty is challenging in terms of dealing with uncertainties and variations.

The traditional Economic Order Quantity (EOQ) and Economic Production Quantity (EPQ) both are widely and successfully used models of inventory management (see Nahmias [1]). In the classical EOQ model the demand occurs at a constant rate, and each time the inventory level hits level 0 an immediate and fixed order of size *Q* is placed. In the classic EPQ model, every production cycle is composed of ON and OFF deterministic periods. There exists a predetermined level, say *Q*, such that the system is ON and the inventory level increases from level 0 up to level *Q*. When level *Q* is reached, the production is stopped and the inventory decreases down to 0. The time it takes from *Q* to 0 is the OFF period. Notice that the EOQ model, in which the items are obtained from an outside suppliers, is just a special case of the EPQ model, by letting the production rate to be almost infinite.

The EOQ model derives the optimum order size that should be placed with a vendor to minimize the holding and ordering costs. On the other hand, the EPQ model determines the optimum production size that is to be manufactured to avoid unnecessary blockage of funds and excess storage costs. Both models consider the timing of reordering or production, the cost incurred due to order/production, and the holding costs to store items; holding cost can further be in the form of rentals for the storage area, salaries of personnel looking after the inventory, electricity bills, repairs, maintenance, etc. Thus, both models describe the trade-off between fixed ordering/producing costs and variable holding costs. Furthermore, both models assume that the demand and production rates are constant over time. The traditional EOQ model assumes that the order arrives with no time, and its replenishment will happen as soon as it reaches the minimum threshold level (usually level 0). Similarly, the traditional EPQ model assumes that the production can starts immediately as the stock goes down below a minimum level (usually level 0). The price is fixed and constant while making a purchase under EOQ or producing under EPQ. The key difference between the two is that the EOQ model is applied when the items are ordered from a third party, and the EPQ model comes into use when the company is the producer itself of the products.

Unfortunately, in real-world, most of the above assumptions are unrealistic; holding and ordering costs may vary due to change in rentals, salaries of personnel, and other overhead expenses. The demand rate, as well as the price of a product, can hardly be constant. They fluctuate a lot in the real world. Consumer income, tastes, and preferences, prices of inputs and raw materials, seasonal variation in demand, etc. are key factors that will affect demand as well as price. Similarly, under the EPQ model, the production process also does not remain constant because of factors like an interruption in power supply, breakages, and repairs in plant and machinery, overheating, change in the quality of inputs and raw materials, etc.

Moreover, as a consequence of home shopping, changes in customer preferences, technological advancements, and competition, modern sales and production companies often offer a take-back guarantee. Companies soften customers' risk by offering a trial period for their products, thus, policies such as the right to return goods have become a part of daily routine. As a result, companies realize that a better understanding of returning items can provide a competitive advantage (Beltran and Krass [2], Fleischmann *et al.* [3], Pinçe *et al.* [4], Shaharudin *et al.* [5], and Barron [6, 7]).

The real need for guidance on how best to handle these uncertainties in demands, returns, productions, and costs motivates this study. We consider a continuous stochastic fluid inventory model for a single-item infinite horizon. Our main focus is to provide contributions to the study of inventory systems modulated by a Markovian environment. In the literature, dynamic control of stochastic inventory systems have been classified as periodic review models and continuous review models. In the case of continuous model, the inventory level (i.e., the number of on-hand items) can be viewed as a fluid process in which the production and demand rates undergo recurring changes in a stochastic fashion, and may be modeled as Markovian.

Markov-modulated fluid flows models have been an active area of research in recent years; one of their main applications is to the modeling the traffic evolution in communication channels. A standard example of a fluid flow is given by an infinite capacity buffer with inflow and outflow rates controlled by a Markov chain. The buffer level increases or decreases linearly at the current rate; when the buffer becomes empty, several strategies can be applied; it can remain empty until the inventory content level reaches a certain barrier (see, e.g., Boxma *et al.* [8], Kulkarni and Yan [9], Bean and O'reilly [10], and Barron and Hermel [11], Baek *et al.* [12], and Baek *et al.* [13]) or it can have positive jumps at the boundary (see e.g. Kulkarni and Yan [9, 14], and Barron [6, 15]). Fluid flow models are appropriate in situations where the arrival is comprised of a discrete unit, but the inter-arrival time between successive arrivals is negligible. Therefore, the arrival can be approximated by a continuous flow of fluid as individual units have less impact on the performance of the system. Such fluid queues are used as modeling tools of high-speed communication networks, transportation systems, congestion control systems, risk processes, and production-inventory systems.

In this study, the on-hand inventory level ℐ ¼ f g *I t*ð Þ : *t*≥ 0 increases or decreases according to a fluid-flow rate modulated by an *n*-state Continuous-Time Markov Chain (CTMC). The fluid process is the inventory position or inventory level under continuous review where the environment process represents the varying background state. A jump in the fluid level represents an external order arrival, and the transition at the background state can be the result of repairs or production facilities, etc. The cost structure includes an ordering cost for each order, a variable cost that is proportional to the actually replenished amount (both for EOQ), a set up cost for production line initialization, a production cost per item (both for EPQ), a holding cost per unit of inventory during time unit, and a penalty cost in case of shortage.

Due to the complexity of the optimal policy, these inventory/production fluid systems are challenging to optimize, and great effort in the past focused on constructing various heuristic policies (Mohebbi [16], Kouki *et al.* [17], Barron [18], and Barron and Dreyfuss [19]). Fluid versions of the EOQ model are studied in Kulkarni and Yan [9], Yan and Kulkarni [20], Kulkarni [21], Berman *et al.* [22], Berman *et al.* [23], Berman and Perry *et al.* [24], and Barron [15] and the references therein. We also mention another related model, so-called clearing system (see Kella *et al.* [25], Berman *et al.* [26], and Barron [27]), which can be regarded as a dual EOQ stochastic model. In a clearing system, the fluid process jumps back to zero when it reaches a certain positive level. For background on stochastic EPQ models, we cite Vickson [28], Kella and Whitt [29], Boxma et al. [8, 30], and Barron [7] among others.

In this chapter, our main objective is to minimize the expected discounted total cost using a discount factor *β* >0. For that, we develop techniques enabling us to determine all the costs in such vendor-managing-inventory models in a closed-form. Our analysis is based on a combination of a certain martingale technique and an application of fluid flow theory to semi-regenerative processes. The martingale

approach was introduced by Asmussen and Kella [31] and was frequently used in the study of inventory models (see, e.g., Boxma *et al.* [8], Kella *et al.* [25], and Barron *et al.* [27, 32]). The matrix-analytic approach and the theory of Markov-modulated fluid flows was initiated by Ramaswami [33] and Ahn *et al*. [34], who developed a unified methodology for studying a large class of insurance risk models via fluid flows by making use of the connection between the surplus process of an insurance and a particular fluid flow.

As we will show, the exit-time results are used to efficiently derive LST (Laplace– Stieltjes transform) functionals associated with the discrete-type measures, while the combination with the martingales yields simple expressions for the continuous-type measures. These explicit expressions can then be used for an analysis of the dependence of the cost functionals on the system parameters or for optimization purpose when some of these parameters (e.g. the order amount, the threshold levels, or the costs) are taken as decision variables. As such, we provide managers with a useful framework and an efficient and easy-to-implement tool to derive the best parameters and to compare the results of different demand–supply patterns.

We start by introducing the main tools of our analysis to be used.
