Showing 1–8 of 8 results for author: Santos, L P

Reducing the Resources Required by ADAPT-VQE Using Coupled Exchange Operators and Improved Subroutines

Authors: Mafalda Ramôa, Panagiotis G. Anastasiou, Luis Paulo Santos, Nicholas J. Mayhall, Edwin Barnes, Sophia E. Economou

Abstract: Adaptive variational quantum algorithms arguably offer the best prospects for quantum advantage in the NISQ era. Since the inception of the first such algorithm, ADAPT-VQE, many improvements have appeared in the literature. We combine the key improvements along with a novel operator pool -- which we term Coupled Exchange Operator (CEO) pool -- to assess the cost of running state-of-the-art ADAPT-V… ▽ More Adaptive variational quantum algorithms arguably offer the best prospects for quantum advantage in the NISQ era. Since the inception of the first such algorithm, ADAPT-VQE, many improvements have appeared in the literature. We combine the key improvements along with a novel operator pool -- which we term Coupled Exchange Operator (CEO) pool -- to assess the cost of running state-of-the-art ADAPT-VQE on hardware in terms of measurement counts and circuit depth. We show a dramatic reduction of these quantum resources compared to the early versions of the algorithm. We also find that our state-of-the-art CEO-ADAPT-VQE outperforms UCCSD, the most widely regarded static VQE ansatz, in all relevant metrics. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Trainability issues in quantum policy gradients

Authors: André Sequeira, Luis Paulo Santos, Luis Soares Barbosa

Abstract: This research explores the trainability of Parameterized Quantum circuit-based policies in Reinforcement Learning, an area that has recently seen a surge in empirical exploration. While some studies suggest improved sample complexity using quantum gradient estimation, the efficient trainability of these policies remains an open question. Our findings reveal significant challenges, including standa… ▽ More This research explores the trainability of Parameterized Quantum circuit-based policies in Reinforcement Learning, an area that has recently seen a surge in empirical exploration. While some studies suggest improved sample complexity using quantum gradient estimation, the efficient trainability of these policies remains an open question. Our findings reveal significant challenges, including standard Barren Plateaus with exponentially small gradients and gradient explosion. These phenomena depend on the type of basis-state partitioning and map** these partitions onto actions. For a polynomial number of actions, a trainable window can be ensured with a polynomial number of measurements if a contiguous-like partitioning of basis-states is employed. These results are empirically validated in a multi-armed bandit environment. △ Less

Submitted 13 June, 2024; originally announced June 2024.

VQC-Based Reinforcement Learning with Data Re-uploading: Performance and Trainability

Authors: Rodrigo Coelho, André Sequeira, Luís Paulo Santos

Abstract: Reinforcement Learning (RL) consists of designing agents that make intelligent decisions without human supervision. When used alongside function approximators such as Neural Networks (NNs), RL is capable of solving extremely complex problems. Deep Q-Learning, a RL algorithm that uses Deep NNs, achieved super-human performance in some specific tasks. Nonetheless, it is also possible to use Variatio… ▽ More Reinforcement Learning (RL) consists of designing agents that make intelligent decisions without human supervision. When used alongside function approximators such as Neural Networks (NNs), RL is capable of solving extremely complex problems. Deep Q-Learning, a RL algorithm that uses Deep NNs, achieved super-human performance in some specific tasks. Nonetheless, it is also possible to use Variational Quantum Circuits (VQCs) as function approximators in RL algorithms. This work empirically studies the performance and trainability of such VQC-based Deep Q-Learning models in classic control benchmark environments. More specifically, we research how data re-uploading affects both these metrics. We show that the magnitude and the variance of the gradients of these models remain substantial throughout training due to the moving targets of Deep Q-Learning. Moreover, we empirically show that increasing the number of qubits does not lead to an exponential vanishing behavior of the magnitude and variance of the gradients for a PQC approximating a 2-design, unlike what was expected due to the Barren Plateau Phenomenon. This hints at the possibility of VQCs being specially adequate for being used as function approximators in such a context. △ Less

Submitted 21 January, 2024; originally announced January 2024.

Comments: 21 pages, 8 figures

On Quantum Natural Policy Gradients

Authors: André Sequeira, Luis Paulo Santos, Luis Soares Barbosa

Abstract: This research delves into the role of the quantum Fisher Information Matrix (FIM) in enhancing the performance of Parameterized Quantum Circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FIM in contextual bandits, its impact in broader reinforcement learning contexts, such as Markov Deci… ▽ More This research delves into the role of the quantum Fisher Information Matrix (FIM) in enhancing the performance of Parameterized Quantum Circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FIM in contextual bandits, its impact in broader reinforcement learning contexts, such as Markov Decision Processes, is less clear. Through a detailed analysis of Löwner inequalities between quantum and classical FIMs, this study uncovers the nuanced distinctions and implications of using each type of FIM. Our results indicate that a PQC-based agent using the quantum FIM without additional insights typically incurs a larger approximation error and does not guarantee improved performance compared to the classical FIM. Empirical evaluations in classic control benchmarks suggest even though quantum FIM preconditioning outperforms standard gradient ascent, in general it is not superior to classical FIM preconditioning. △ Less

Submitted 16 January, 2024; originally announced January 2024.

Reducing measurement costs by recycling the Hessian in adaptive variational quantum algorithms

Authors: Mafalda Ramôa, Luis Paulo Santos, Nicholas J. Mayhall, Edwin Barnes, Sophia E. Economou

Abstract: Adaptive protocols enable the construction of more efficient state preparation circuits in variational quantum algorithms (VQAs) by utilizing data obtained from the quantum processor during the execution of the algorithm. This idea originated with ADAPT-VQE, an algorithm that iteratively grows the state preparation circuit operator by operator, with each new operator accompanied by a new variation… ▽ More Adaptive protocols enable the construction of more efficient state preparation circuits in variational quantum algorithms (VQAs) by utilizing data obtained from the quantum processor during the execution of the algorithm. This idea originated with ADAPT-VQE, an algorithm that iteratively grows the state preparation circuit operator by operator, with each new operator accompanied by a new variational parameter, and where all parameters acquired thus far are optimized in each iteration. In ADAPT-VQE and other adaptive VQAs that followed it, it has been shown that initializing parameters to their optimal values from the previous iteration speeds up convergence and avoids shallow local traps in the parameter landscape. However, no other data from the optimization performed at one iteration is carried over to the next. In this work, we propose an improved quasi-Newton optimization protocol specifically tailored to adaptive VQAs. The distinctive feature in our proposal is that approximate second derivatives of the cost function are recycled across iterations in addition to parameter values. We implement a quasi-Newton optimizer where an approximation to the inverse Hessian matrix is continuously built and grown across the iterations of an adaptive VQA. The resulting algorithm has the flavor of a continuous optimization where the dimension of the search space is augmented when the gradient norm falls below a given threshold. We show that this inter-optimization exchange of second-order information leads the Hessian in the state of the optimizer to better approximate the exact Hessian. As a result, our method achieves a superlinear convergence rate even in situations where the typical quasi-Newton optimizer converges only linearly. Our protocol decreases the measurement costs in implementing adaptive VQAs on quantum hardware as well as the runtime of their classical simulation. △ Less

Submitted 10 January, 2024; originally announced January 2024.

Towards Quantum Ray Tracing

Authors: Luís Paulo Santos, Thomas Bashford-Rogers, João Barbosa, Paul Navrátil

Abstract: Rendering on conventional computers is capable of generating realistic imagery, but the computational complexity of these light transport algorithms is a limiting factor of image synthesis. Quantum computers have the potential to significantly improve rendering performance through reducing the underlying complexity of the algorithms behind light transport. This paper investigates hybrid quantum-cl… ▽ More Rendering on conventional computers is capable of generating realistic imagery, but the computational complexity of these light transport algorithms is a limiting factor of image synthesis. Quantum computers have the potential to significantly improve rendering performance through reducing the underlying complexity of the algorithms behind light transport. This paper investigates hybrid quantum-classical algorithms for ray tracing, a core component of most rendering techniques. Through a practical implementation of quantum ray tracing in a 3D environment, we show quantum approaches provide a quadratic improvement in query complexity compared to the equivalent classical approach. Based on domain specific knowledge, we then propose algorithms to significantly reduce the computation required for quantum ray tracing through exploiting image space coherence and a principled termination criteria for quantum searching. We show results for both Whitted style ray tracing, and for accelerating ray tracing operations when performing classical Monte Carlo integration for area lights and indirect illumination. △ Less

Submitted 27 April, 2022; originally announced April 2022.

Comments: 27 pages, 15 figures

MSC Class: 68U05 ACM Class: I.3.7

Policy Gradients using Variational Quantum Circuits

Authors: André Sequeira, Luis Paulo Santos, Luís Soares Barbosa

Abstract: Variational Quantum Circuits are being used as versatile Quantum Machine Learning models. Some empirical results exhibit an advantage in supervised and generative learning tasks. However, when applied to Reinforcement Learning, less is known. In this work, we considered a Variational Quantum Circuit composed of a low-depth hardware-efficient ansatz as the parameterized policy of a Reinforcement Le… ▽ More Variational Quantum Circuits are being used as versatile Quantum Machine Learning models. Some empirical results exhibit an advantage in supervised and generative learning tasks. However, when applied to Reinforcement Learning, less is known. In this work, we considered a Variational Quantum Circuit composed of a low-depth hardware-efficient ansatz as the parameterized policy of a Reinforcement Learning agent. We show that an $ε$-approximation of the policy gradient can be obtained using a logarithmic number of samples concerning the total number of parameters. We empirically verify that such quantum models behave similarly or even outperform typical classical neural networks used in standard benchmarking environments and in quantum control, using only a fraction of the parameters. Moreover, we study the Barren Plateau phenomenon in quantum policy gradients using the Fisher Information Matrix spectrum. △ Less

Submitted 15 January, 2023; v1 submitted 20 March, 2022; originally announced March 2022.

Generalised Quantum Tree Search

Authors: Andre Sequeira, Luis Paulo Santos, Luis Soares Barbosa

Abstract: This extended abstract reports on on-going research on quantum algorithmic approaches to the problem of generalised tree search that may exhibit effective quantum speedup, even in the presence of non-constant branching factors. Two strategies are briefly summarised and current work outlined. This extended abstract reports on on-going research on quantum algorithmic approaches to the problem of generalised tree search that may exhibit effective quantum speedup, even in the presence of non-constant branching factors. Two strategies are briefly summarised and current work outlined. △ Less

Submitted 25 March, 2021; originally announced March 2021.