Industrializing deep reinforcement learning for operational spare parts inventory management
van der Haar, Joost F van Jaarsveld, Willem Basten, Rob JI Boute, Robert N
van der Haar, Joost F
van Jaarsveld, Willem
Basten, Rob JI
Boute, Robert N
Publication Type
Journal article with impact factor
Editor
Supervisor
Publication Year
2026-10
Journal
European Journal of Operational Research
Book
Publication Volume
334
Publication Issue
1
Publication Begin page
Publication End page
Publication Number of pages
Collections
Abstract
We show how Deep Reinforcement Learning (DRL) can improve industrial-scale operational spare parts inventory management. Spare parts inventory is crucial for the timely maintenance of capital goods. Operational spare parts management requires fast decision-making for complex and large-scale service networks. Therefore, existing work focuses on computationally-light heuristics to maintain tractability, even if that comes at a cost in performance. This is problematic, as lower performance may mean unnecessary downtime on a bottleneck machine, or excessive spending on inventory procurement and expedited shipments. Previous studies have shown that DRL models can be trained to take high-quality decisions for complex, albeit small, inventory problems almost instantaneously. Nonetheless, training DRL models for industrial-scale inventory systems remains an open challenge. We propose a novel DRL approach that adapts and improves upon three techniques: action space decomposition to cope with a large number of locations, cross-SKU learning to scale over an arbitrary number of SKUs, and reward smoothing to efficiently handle stochastic and sparse demand. We demonstrate our approach’s effectiveness on the service network of ASML, a leading company in the semiconductor industry. The results show that our DRL approach outperforms existing methods on a fully-connected service network with 10,000 SKUs and 60 locations by making intelligent use of distance-based proactive lateral transshipments and expediting. This policy insight has allowed ASML to adapt its existing algorithm, leading to a more than 10% reduction in critical material shortages.
Research Projects
Organizational Units
Journal Issue
Keywords
46 Information and Computing Sciences, 4611 Machine Learning, Machine Learning and Artificial Intelligence, 9 Industry, Innovation and Infrastructure