Browsing by Author "Boute, Robert N."
Now showing 1 - 6 of 6
Results Per Page
Sort Options
- Can deep reinforcement learning improve inventory management? Performance on lost sales, dual-sourcing, and multi-echelon problemsPublication . Gijsbrechts, Joren; Boute, Robert N.; Mieghem, Jan A. van; Zhang, Dennis J.Problem definition: Is deep reinforcement learning (DRL) effective at solving inventory problems? Academic/practical relevance: Given that DRL has successfully been applied in computer games and robotics, supply chain researchers and companies are interested in its potential in inventory management. We provide a rigorous performance evaluation of DRL in three classic and intractable inventory problems: lost sales, dual sourcing, and multi-echelon inventory management. Methodology: We model each inventory problem as a Markov decision process and apply and tune the Asynchronous Advantage Actor-Critic (A3C) DRL algorithm for a variety of parameter settings. Results: We demonstrate that the A3C algorithm can match the performance of the state-of-the-art heuristics and other approximate dynamic programming methods. Although the initial tuning was computationally demanding and time demanding, only small changes to the tuning parameters were needed for the other studied problems. Managerial implications: Our study provides evidence that DRL can effectively solve stationary inventory problems. This is especially promising when problem-dependent heuristics are lacking. Yet, generating structural policy insight or designing specialized policies that are (ideally provably) near optimal remains desirable.
- Deep reinforcement learning for inventory control: a roadmapPublication . Boute, Robert N.; Gijsbrechts, Joren; Jaarsveld, Willem van; Vanvuchelen, NathalieDeep reinforcement learning (DRL) has shown great potential for sequential decision-making, including early developments in inventory control. Yet, the abundance of choices that come with designing a DRL algorithm, combined with the intense computational effort to tune and evaluate each choice, may hamper their application in practice. This paper describes the key design choices of DRL algorithms to facilitate their implementation in inventory control. We also shed light on possible future research avenues that may elevate the current state-of-the-art of DRL applications for inventory control and broaden their scope by leveraging and improving on the structural policy insights within inventory research. Our discussion and roadmap may also spur future research in other domains within operations management.
- Dual sourcing and smoothing under nonstationary demand time series: reshoring with SpeedFactoriesPublication . Boute, Robert N.; Disney, Stephen M.; Gijsbrechts, Joren; Mieghem, Jan A. VanWe investigate near-shoring a small part of the global production to local SpeedFactories that serve only the variable demand. The short lead time of the responsive SpeedFactory reduces the risk of making large volumes in advance, yet it does not involve a complete reshoring of demand. Using a break-even analysis, we investigate the lead time, demand, and cost characteristics that make dual sourcing with a SpeedFactory desirable compared with complete off-shoring. Our analysis uses a linear generalization of the celebrated order-up-to inventory policy to settings where capacity costs exist. The policy allows for order smoothing to reduce capacity costs and performs well relative to the (unknown) optimal policy. We highlight the significant impact of auto-correlated and nonstationary demand series, which are prevalent in practice yet challenging to analyze, on the economic benefit of reshoring. Methodologically, we adopt a linear policy and normally distributed demand and use Zβtransforms to present exact analyses.
- Optimal robust inventory management with volume flexibility: matching capacity and demand with the lookahead peak-shaving policyPublication . Gijsbrechts, Joren; Imdahl, Christina; Boute, Robert N.; Van Mieghem, Jan A.We study inventory control with volume flexibility: A firm can replenish using period-dependent base capacity at regular sourcing costs and access additional supply at a premium. The optimal replenishment policy is characterized by two period-dependent base-stock levels but determining their values is not trivial, especially for nonstationary and correlated demand. We propose the Lookahead Peak-Shaving policy that anticipates and peak shaves orders from future peak-demand periods to the current period, thereby matching capacity and demand. Peak shaving anticipates future order peaks and partially shifts them forward. This contrasts with conventional smoothing, which recovers the inventory deficit resulting from demand peaks by increasing later orders. Our contribution is threefold. First, we use a novel iterative approach to prove the robust optimality of the Lookahead Peak-Shaving policy. Second, we provide explicit expressions of the period-dependent base-stock levels and analyze the amount of peak shaving. Finally, we demonstrate how our policy outperforms other heuristics in stochastic systems. Most cost savings occur when demand is nonstationary and negatively correlated, and base capacities fluctuate around the mean demand. Our insights apply to several practical settings, including production systems with overtime, sourcing from multiple capacitated suppliers, or transportation planning with a spot market. Applying our model to data from a manufacturer reduces inventory and sourcing costs by 6.7%, compared to the manufacturer's policy without peak shaving.
- Reward shaping to improve the performance of deep reinforcement learning in perishable inventory managementPublication . Moor, Bram J. de; Gijsbrechts, Joren; Boute, Robert N.Deep reinforcement learning (DRL) has proven to be an effective, general-purpose technology to develop βgoodβ replenishment policies in inventory management. We show how transfer learning from existing, well-performing heuristics may stabilize the training process and improve the performance of DRL in inventory control. While the idea is general, we specifically implement potential-based reward shaping to a deep Q-network algorithm to manage inventory of perishable goods that, cursed by dimensionality, has proven to be notoriously complex. The application of our approach may not only improve inventory cost performance and reduce computational effort, the increased training stability may also help to gain trust in the policies obtained by black box DRL algorithms.
- Volume flexibility at responsive suppliers in reshoring decisions: analysis of a dual sourcing inventory modelPublication . Gijsbrechts, Joren; Boute, Robert N.; Disney, Stephen M.; Mieghem, Jan A. VanWe investigate how volume flexibility, defined by a sourcing cost premium beyond a base capacity, at a local responsive supplier impacts the decision to reshore supply. The buyer also has access to a remote supplier that is cheaper with no restrictions on volume flexibility. We show that with unit lead time difference between both suppliers, the optimal dual sourcing policy is a modified dual base-stock policy with three base-stock levels ππ1, ππ2, and ππ . The replenishment orders are generated by first placing a base order from the fast supplier of at most π units to raise the inventory position to ππ1, if that is possible. After this base order, if the adjusted inventory position is still below ππ2, additional units are ordered from the fast supplier at an overtime premium to reach ππ2. Finally, if the adjusted inventory position is below ππ , an order from the slow supplier is placed to bring the final inventory position to ππ . Surprisingly, in contrast to single sourcing with limited volume flexibility, a more complex dual sourcing model often results in a βsimplerβ policy that replaces demand in each period. The latter allows analytical insights into the sourcing split between the responsive and the remote supplier. Our analysis shows how increased volume flexibility at the responsive supplier promotes the decision to reshore operations and effectively serves as a cost benefit. It also shows how investing in base capacity or additional volume flexibility act as strategic substitutes.