-
BCQQ: Batch-Constraint Quantum Q-Learning with Cyclic Data Re-uploading
Authors:
Maniraman Periyasamy,
Marc Hölle,
Marco Wiedmann,
Daniel D. Scherer,
Axel Plinge,
Christopher Mutschler
Abstract:
Deep reinforcement learning (DRL) often requires a large number of data and environment interactions, making the training process time-consuming. This challenge is further exacerbated in the case of batch RL, where the agent is trained solely on a pre-collected dataset without environment interactions. Recent advancements in quantum computing suggest that quantum models might require less data for…
▽ More
Deep reinforcement learning (DRL) often requires a large number of data and environment interactions, making the training process time-consuming. This challenge is further exacerbated in the case of batch RL, where the agent is trained solely on a pre-collected dataset without environment interactions. Recent advancements in quantum computing suggest that quantum models might require less data for training compared to classical methods. In this paper, we investigate this potential advantage by proposing a batch RL algorithm that utilizes VQC as function approximators within the discrete batch-constraint deep Q-learning (BCQ) algorithm. Additionally, we introduce a novel data re-uploading scheme by cyclically shifting the order of input variables in the data encoding layers. We evaluate the efficiency of our algorithm on the OpenAI CartPole environment and compare its performance to the classical neural network-based discrete BCQ.
△ Less
Submitted 18 March, 2024; v1 submitted 27 April, 2023;
originally announced May 2023.
-
An Empirical Comparison of Optimizers for Quantum Machine Learning with SPSA-based Gradients
Authors:
Marco Wiedmann,
Marc Hölle,
Maniraman Periyasamy,
Nico Meyer,
Christian Ufrecht,
Daniel D. Scherer,
Axel Plinge,
Christopher Mutschler
Abstract:
VQA have attracted a lot of attention from the quantum computing community for the last few years. Their hybrid quantum-classical nature with relatively shallow quantum circuits makes them a promising platform for demonstrating the capabilities of NISQ devices. Although the classical machine learning community focuses on gradient-based parameter optimization, finding near-exact gradients for VQC w…
▽ More
VQA have attracted a lot of attention from the quantum computing community for the last few years. Their hybrid quantum-classical nature with relatively shallow quantum circuits makes them a promising platform for demonstrating the capabilities of NISQ devices. Although the classical machine learning community focuses on gradient-based parameter optimization, finding near-exact gradients for VQC with the parameter-shift rule introduces a large sampling overhead. Therefore, gradient-free optimizers have gained popularity in quantum machine learning circles. Among the most promising candidates is the SPSA algorithm, due to its low computational cost and inherent noise resilience. We introduce a novel approach that uses the approximated gradient from SPSA in combination with state-of-the-art gradient-based classical optimizers. We demonstrate numerically that this outperforms both standard SPSA and the parameter-shift rule in terms of convergence rate and absolute error in simple regression tasks. The improvement of our novel approach over SPSA with stochastic gradient decent is even amplified when shot- and hardware-noise are taken into account. We also demonstrate that error mitigation does not significantly affect our results.
△ Less
Submitted 27 April, 2023;
originally announced May 2023.
-
Out-of-equilibrium operation of a quantum heat engine: The cost of thermal coupling control
Authors:
Michael Wiedmann,
Jürgen T. Stockburger,
Joachim Ankerhold
Abstract:
Real quantum heat engines lack the separation of time and length scales that is characteristic for classical engines. They must be understood as open quantum systems in non-equilibrium with time-controlled coupling to thermal reservoirs as integral part. Here, we present a systematic approach to describe a broad class of engines and protocols beyond conventional weak coupling treatments starting f…
▽ More
Real quantum heat engines lack the separation of time and length scales that is characteristic for classical engines. They must be understood as open quantum systems in non-equilibrium with time-controlled coupling to thermal reservoirs as integral part. Here, we present a systematic approach to describe a broad class of engines and protocols beyond conventional weak coupling treatments starting from a microscopic modeling. For the four stroke Otto engine the full dynamical range down to low temperatures is explored and the crucial role of the work associated with the coupling/decoupling to/from reservoirs in the energy balance is revealed. Quantum correlations turn out to be instrumental to enhance the efficiency which opens new ways for optimal control techniques.
△ Less
Submitted 27 September, 2019; v1 submitted 27 March, 2019;
originally announced March 2019.
-
Rectification of heat currents across nonlinear quantum chains: A versatile approach beyond weak thermal contact
Authors:
Thomas Motz,
Michael Wiedmann,
Jürgen T. Stockburger,
Joachim Ankerhold
Abstract:
Within the emerging field of quantum thermodynamics the issues of heat transfer and heat rectification are basic ingredients for the understanding and design of heat engines or refrigerators at nanoscales. Here, a consistent and versatile approach for mesoscopic devices operating with continuous degrees of freedom is developed valid from low up to strong system-reservoir couplings and over the who…
▽ More
Within the emerging field of quantum thermodynamics the issues of heat transfer and heat rectification are basic ingredients for the understanding and design of heat engines or refrigerators at nanoscales. Here, a consistent and versatile approach for mesoscopic devices operating with continuous degrees of freedom is developed valid from low up to strong system-reservoir couplings and over the whole temperature range. It allows to cover weak to moderate nonlinearities and is applicable to various scenarios including the presence of disorder and external time-dependent fields. As a particular application coherent one-dimensional chains of anharmonic oscillators terminated by thermal reservoirs are analyzed with particular focus on rectification. The efficiency of the method opens a door to treat also rather long chains and extensions to higher dimensions and geometries.
△ Less
Submitted 22 October, 2018; v1 submitted 19 July, 2018;
originally announced July 2018.
-
Time-Correlated Blip Dynamics of Open Quantum Systems
Authors:
Michael Wiedmann,
Jürgen T. Stockburger,
Joachim Ankerhold
Abstract:
The non-Markovian dynamics of open quantum systems is still a challenging task, particularly in the non-perturbative regime at low temperatures. While the Stochastic Liouville-von Neumann equation (SLN) provides a formally exact tool to tackle this problem for both discrete and continuous degrees of freedom, its performance deteriorates for long times due to an inherently non-unitary propagator. H…
▽ More
The non-Markovian dynamics of open quantum systems is still a challenging task, particularly in the non-perturbative regime at low temperatures. While the Stochastic Liouville-von Neumann equation (SLN) provides a formally exact tool to tackle this problem for both discrete and continuous degrees of freedom, its performance deteriorates for long times due to an inherently non-unitary propagator. Here we present a scheme which combines the SLN with projector operator techniques based on finite dephasing times, gaining substantial improvements in terms of memory storage and statistics. The approach allows for systematic convergence and is applicable in regions of parameter space where perturbative methods fail, up to the long time domain. Findings are applied to the coherent and incoherent quantum dynamics of two- and three-level systems. In the long time domain sequential and super-exchange transfer rates are extracted and compared to perturbative predictions.
△ Less
Submitted 28 September, 2016;
originally announced September 2016.
