-
Deep Reinforcement Learning for Multi-Truck Vehicle Routing Problems with Multi-Leg Demand Routes
Authors:
Joshua Levin,
Randall Correll,
Takanori Ide,
Takafumi Suzuki,
Takaho Saito,
Alan Arai
Abstract:
Deep reinforcement learning (RL) has been shown to be effective in producing approximate solutions to some vehicle routing problems (VRPs), especially when using policies generated by encoder-decoder attention mechanisms. While these techniques have been quite successful for relatively simple problem instances, there are still under-researched and highly complex VRP variants for which no effective…
▽ More
Deep reinforcement learning (RL) has been shown to be effective in producing approximate solutions to some vehicle routing problems (VRPs), especially when using policies generated by encoder-decoder attention mechanisms. While these techniques have been quite successful for relatively simple problem instances, there are still under-researched and highly complex VRP variants for which no effective RL method has been demonstrated. In this work we focus on one such VRP variant, which contains multiple trucks and multi-leg routing requirements. In these problems, demand is required to move along sequences of nodes, instead of just from a start node to an end node. With the goal of making deep RL a viable strategy for real-world industrial-scale supply chain logistics, we develop new extensions to existing encoder-decoder attention models which allow them to handle multiple trucks and multi-leg routing requirements. Our models have the advantage that they can be trained for a small number of trucks and nodes, and then embedded into a large supply chain to yield solutions for larger numbers of trucks and nodes. We test our approach on a real supply chain environment arising in the operations of Japanese automotive parts manufacturer Aisin Corporation, and find that our algorithm outperforms Aisin's previous best solution.
△ Less
Submitted 8 January, 2024;
originally announced January 2024.
-
Quantum Neural Networks for a Supply Chain Logistics Application
Authors:
Randall Correll,
Sean J. Weinberg,
Fabio Sanches,
Takanori Ide,
Takafumi Suzuki
Abstract:
Problem instances of a size suitable for practical applications are not likely to be addressed during the noisy intermediate-scale quantum (NISQ) period with (almost) pure quantum algorithms. Hybrid classical-quantum algorithms have potential, however, to achieve good performance on much larger problem instances. We investigate one such hybrid algorithm on a problem of substantial importance: vehi…
▽ More
Problem instances of a size suitable for practical applications are not likely to be addressed during the noisy intermediate-scale quantum (NISQ) period with (almost) pure quantum algorithms. Hybrid classical-quantum algorithms have potential, however, to achieve good performance on much larger problem instances. We investigate one such hybrid algorithm on a problem of substantial importance: vehicle routing for supply chain logistics with multiple trucks and complex demand structure. We use reinforcement learning with neural networks with embedded quantum circuits. In such neural networks, projecting high-dimensional feature vectors down to smaller vectors is necessary to accommodate restrictions on the number of qubits of NISQ hardware. However, we use a multi-head attention mechanism where, even in classical machine learning, such projections are natural and desirable. We consider data from the truck routing logistics of a company in the automotive sector, and apply our methodology by decomposing into small teams of trucks, and we find results comparable to human truck assignment.
△ Less
Submitted 2 December, 2022; v1 submitted 30 November, 2022;
originally announced December 2022.
-
Reinforcement Learning for Multi-Truck Vehicle Routing Problems
Authors:
Randall Correll,
Sean J. Weinberg,
Fabio Sanches,
Takanori Ide,
Takafumi Suzuki
Abstract:
Vehicle routing problems and other combinatorial optimization problems have been approximately solved by reinforcement learning agents with policies based on encoder-decoder models with attention mechanisms. These techniques are of substantial interest but still cannot solve the complex routing problems that arise in a realistic setting which can have many trucks and complex requirements. With the…
▽ More
Vehicle routing problems and other combinatorial optimization problems have been approximately solved by reinforcement learning agents with policies based on encoder-decoder models with attention mechanisms. These techniques are of substantial interest but still cannot solve the complex routing problems that arise in a realistic setting which can have many trucks and complex requirements. With the aim of making reinforcement learning a viable technique for supply chain optimization, we develop new extensions to encoder-decoder models for vehicle routing that allow for complex supply chains using classical computing today and quantum computing in the future. We make two major generalizations. First, our model allows for routing problems with multiple trucks. Second, we move away from the simple requirement of having a truck deliver items from nodes to one special depot node, and instead allow for a complex tensor demand structure. We show how our model, even if trained only for a small number of trucks, can be embedded into a large supply chain to yield viable solutions.
△ Less
Submitted 10 December, 2022; v1 submitted 30 November, 2022;
originally announced November 2022.
-
Supply Chain Logistics with Quantum and Classical Annealing Algorithms
Authors:
Sean J. Weinberg,
Fabio Sanches,
Takanori Ide,
Kazumitzu Kamiya,
Randall Correll
Abstract:
Noisy intermediate-scale quantum (NISQ) hardware is almost universally incompatible with full-scale optimization problems of practical importance which can have many variables and unwieldy objective functions. As a consequence, there is a growing body of literature that tests quantum algorithms on miniaturized versions of problems that arise in an operations research setting. Rather than taking th…
▽ More
Noisy intermediate-scale quantum (NISQ) hardware is almost universally incompatible with full-scale optimization problems of practical importance which can have many variables and unwieldy objective functions. As a consequence, there is a growing body of literature that tests quantum algorithms on miniaturized versions of problems that arise in an operations research setting. Rather than taking this approach, we investigate a problem of substantial commercial value, multi-truck vehicle routing for supply chain logistics, at the scale used by a corporation in their operations. Such a problem is too complex to be fully embedded on any near-term quantum hardware or simulator; we avoid confronting this challenge by taking a hybrid workflow approach: we iteratively assign routes for trucks by generating a new binary optimization problem instance one truck at a time. Each instance has $\sim 2500$ quadratic binary variables, putting it in a range that is feasible for NISQ quantum computing, especially quantum annealing hardware. We test our methods using simulated annealing and the D-Wave Hybrid solver as a place-holder in wait of quantum hardware developments. After feeding the vehicle routes suggested by these runs into a highly realistic classical supply chain simulation, we find excellent performance for the full supply chain. Our work gives a set of techniques that can be adopted in contexts beyond vehicle routing to apply NISQ devices in a hybrid fashion to large-scale problems of commercial interest.
△ Less
Submitted 9 May, 2022;
originally announced May 2022.
-
An Efficient User-Side Nulling Calibration for Quantum Annealing Computers
Authors:
Randall R. Correll
Abstract:
(Withdrawn) This work describes an efficient user-side method of calibrating and correcting quantum annealing computers. For quantum annealing computers based on the Ising model, the method measures the residual bias of the h and J coefficients. Once measured, these biases can then be nulled in subsequent runs for any problem of interest. This method also returns a temperature for each qubit based…
▽ More
(Withdrawn) This work describes an efficient user-side method of calibrating and correcting quantum annealing computers. For quantum annealing computers based on the Ising model, the method measures the residual bias of the h and J coefficients. Once measured, these biases can then be nulled in subsequent runs for any problem of interest. This method also returns a temperature for each qubit based on the measured versus the expected qubit distributions computed from a Boltzmann distribution model.
△ Less
Submitted 2 April, 2015; v1 submitted 2 March, 2015;
originally announced March 2015.
-
Grazing Collisions of Black Holes via the Excision of Singularities
Authors:
Steve Brandt,
Randall Correll,
Roberto Gomez,
Mijan Huq,
Pablo Laguna,
Luis Lehner,
Pedro Marronetti,
Richard A. Matzner,
David Neilsen,
Jorge Pullin,
Erik Schnetter,
Deirdre Shoemaker,
Jeffrey Winicour
Abstract:
We present the first simulations of non-headon (grazing) collisions of binary black holes in which the black hole singularities have been excised from the computational domain. Initially two equal mass black holes $m$ are separated a distance $\approx10m$ and with impact parameter $\approx2m$. Initial data are based on superposed, boosted (velocity $\approx0.5c$) solutions of single black holes…
▽ More
We present the first simulations of non-headon (grazing) collisions of binary black holes in which the black hole singularities have been excised from the computational domain. Initially two equal mass black holes $m$ are separated a distance $\approx10m$ and with impact parameter $\approx2m$. Initial data are based on superposed, boosted (velocity $\approx0.5c$) solutions of single black holes in Kerr-Schild coordinates. Both rotating and non-rotating black holes are considered. The excised regions containing the singularities are specified by following the dynamics of apparent horizons. Evolutions of up to $t \approx 35m$ are obtained in which two initially separate apparent horizons are present for $t\approx3.8m$. At that time a single envelo** apparent horizon forms, indicating that the holes have merged. Apparent horizon area estimates suggest gravitational radiation of about 2.6% of the total mass. The evolutions end after a moderate amount of time because of instabilities.
△ Less
Submitted 14 September, 2000; v1 submitted 13 September, 2000;
originally announced September 2000.
