-
RLOR: A Flexible Framework of Deep Reinforcement Learning for Operation Research
Authors:
Ching Pui Wan,
Tung Li,
Jason Min Wang
Abstract:
Reinforcement learning has been applied in operation research and has shown promise in solving large combinatorial optimization problems. However, existing works focus on develo** neural network architectures for certain problems. These works lack the flexibility to incorporate recent advances in reinforcement learning, as well as the flexibility of customizing model architectures for operation…
▽ More
Reinforcement learning has been applied in operation research and has shown promise in solving large combinatorial optimization problems. However, existing works focus on develo** neural network architectures for certain problems. These works lack the flexibility to incorporate recent advances in reinforcement learning, as well as the flexibility of customizing model architectures for operation research problems. In this work, we analyze the end-to-end autoregressive models for vehicle routing problems and show that these models can benefit from the recent advances in reinforcement learning with a careful re-implementation of the model architecture. In particular, we re-implemented the Attention Model and trained it with Proximal Policy Optimization (PPO) in CleanRL, showing at least 8 times speed up in training time. We hereby introduce RLOR, a flexible framework for Deep Reinforcement Learning for Operation Research. We believe that a flexible framework is key to develo** deep reinforcement learning models for operation research problems. The code of our work is publicly available at https://github.com/cpwan/RLOR.
△ Less
Submitted 23 March, 2023;
originally announced March 2023.
-
An integrated heterogeneous computing framework for ensemble simulations of laser-induced ignition
Authors:
Kazuki Maeda,
Thiago Teixeira,
Jonathan M. Wang,
Jeffrey M. Hokanson,
Caetano Melone,
Mario Di Renzo,
Steve Jones,
Javier Urzay,
Gianluca Iaccarino
Abstract:
An integrated computational framework is introduced to study complex engineering systems through physics-based ensemble simulations on heterogeneous supercomputers. The framework is primarily designed for the quantitative assessment of laser-induced ignition in rocket engines. We develop and combine an implicit programming system, a compressible reacting flow solver, and a data generation/manageme…
▽ More
An integrated computational framework is introduced to study complex engineering systems through physics-based ensemble simulations on heterogeneous supercomputers. The framework is primarily designed for the quantitative assessment of laser-induced ignition in rocket engines. We develop and combine an implicit programming system, a compressible reacting flow solver, and a data generation/management strategy on a robust and portable platform. We systematically present this framework using test problems on a hybrid CPU/GPU machine. Efficiency, scalability, and accuracy of the solver are comprehensively assessed with canonical unit problems. Ensemble data management and autoencoding are demonstrated using a canonical diffusion flame case. Sensitivity analysis of the ignition of a turbulent, gaseous fuel jet is performed using a simplified, three-dimensional model combustor. Our approach unifies computer science, physics and engineering, and data science to realize a cross-disciplinary workflow. The framework is exascale-oriented and can be considered a benchmark for future computational science studies of real-world systems.
△ Less
Submitted 4 February, 2022;
originally announced February 2022.
-
Energy Efficient Medium Access with Interference Mitigation in LTE Femtocell Networks
Authors:
Ying Wang,
Xiangming Dai,
Jason Min Wang,
Brahim Bensaou
Abstract:
With the rapidly increasing number of deployed LTE femtocell base stations (FBS), energy consumption of femtocell networks has become a serious environmental issue. Therefore, energy-efficient protocols are needed to balance the trade-off between energy saving and bandwidth utilization. The key component of the desired protocol to prevent both energy and bandwidth waste is interference mitigation,…
▽ More
With the rapidly increasing number of deployed LTE femtocell base stations (FBS), energy consumption of femtocell networks has become a serious environmental issue. Therefore, energy-efficient protocols are needed to balance the trade-off between energy saving and bandwidth utilization. The key component of the desired protocol to prevent both energy and bandwidth waste is interference mitigation, which, nevertheless, most previous work has failed to properly consider. To this end, in this paper, we manipulate user equipment (UE) association and OFDMA scheduling with a combination of interference mitigation. Recognizing the NP-hardness of the problem, we propose two distributed algorithms with guaranteed convergence. Extensive simulations show that our algorithms outperform the alternative algorithms in multiple metrics such as utility, power consumption, and convergence speed.
△ Less
Submitted 6 August, 2015;
originally announced August 2015.
-
Nano-Sim: A Step Wise Equivalent Conductance based Statistical Simulator for Nanotechnology Circuit Design
Authors:
Bharat Sukhwani,
Uday Padmanabhan,
Janet M. Wang
Abstract:
New nanotechnology based devices are replacing CMOS devices to overcome CMOS technology's scaling limitations. However, many such devices exhibit non-monotonic I-V characteristics and uncertain properties which lead to the negative differential resistance (NDR) problem and the chaotic performance. This paper proposes a new circuit simulation approach that can effectively simulate nanotechnology…
▽ More
New nanotechnology based devices are replacing CMOS devices to overcome CMOS technology's scaling limitations. However, many such devices exhibit non-monotonic I-V characteristics and uncertain properties which lead to the negative differential resistance (NDR) problem and the chaotic performance. This paper proposes a new circuit simulation approach that can effectively simulate nanotechnology devices with uncertain input sources and negative differential resistance (NDR) problem. The experimental results show a 20-30 times speedup comparing with existing simulators.
△ Less
Submitted 25 October, 2007;
originally announced October 2007.
