-
Sample-efficient Model-based Reinforcement Learning for Quantum Control
Authors:
Irtaza Khalid,
Carrie A. Weidner,
Edmond A. Jonckheere,
Sophie G. Shermer,
Frank C. Langbein
Abstract:
We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization with improved sample complexity over model-free RL. Sample complexity is the number of controller interactions with the physical system. Leveraging an inductive bias, inspired by recent advances in neural ordinary differential equations (ODEs), we use an auto-differentiable ODE parametrised by…
▽ More
We propose a model-based reinforcement learning (RL) approach for noisy time-dependent gate optimization with improved sample complexity over model-free RL. Sample complexity is the number of controller interactions with the physical system. Leveraging an inductive bias, inspired by recent advances in neural ordinary differential equations (ODEs), we use an auto-differentiable ODE parametrised by a learnable Hamiltonian ansatz to represent the model approximating the environment whose time-dependent part, including the control, is fully known. Control alongside Hamiltonian learning of continuous time-independent parameters is addressed through interactions with the system. We demonstrate an order of magnitude advantage in the sample complexity of our method over standard model-free RL in preparing some standard unitary gates with closed and open system dynamics, in realistic numerical experiments incorporating single shot measurements, arbitrary Hilbert space truncations and uncertainty in Hamiltonian parameters. Also, the learned Hamiltonian can be leveraged by existing control methods like GRAPE for further gradient-based optimization with the controllers found by RL as initializations. Our algorithm that we apply on nitrogen vacancy (NV) centers and transmons in this paper is well suited for controlling partially characterised one and two qubit systems.
△ Less
Submitted 2 October, 2023; v1 submitted 19 April, 2023;
originally announced April 2023.
-
Robustness of Energy Landscape Control to Dephasing
Authors:
Sean Patrick O'Neil,
Frank C. Langbein,
Edmond Jonckheere,
Sophie Shermer
Abstract:
As shown in previous work, in some cases closed quantum systems exhibit a non-conventional trade-off in performance and robustness in the sense that controllers with the highest fidelity can also provide the best robustness to parameter uncertainty. As the dephasing induced by the interaction of the system with the environment guides the evolution to a more classically mixed state, it is worth inv…
▽ More
As shown in previous work, in some cases closed quantum systems exhibit a non-conventional trade-off in performance and robustness in the sense that controllers with the highest fidelity can also provide the best robustness to parameter uncertainty. As the dephasing induced by the interaction of the system with the environment guides the evolution to a more classically mixed state, it is worth investigating what effect the introduction of dephasing has on the relationship between performance and robustness. In this paper we analyze the robustness of the fidelity error, as measured by the logarithmic sensitivity function, to dephasing processes. We show that introduction of dephasing as a perturbation to the nominal unitary dynamics requires a modification of the log-sensitivity formulation used to measure robustness about an uncertain parameter with non-zero nominal value used in previous work. We consider controllers optimized for a number of target objectives ranging from fidelity under coherent evolution to fidelity under dephasing dynamics to determine the extent to which optimizing for a specific regime has desirable effects in terms of robustness. Our analysis is based on two independent computations of the log-sensitivity: a statistical Monte Carlo approach and an analytic calculation. We show that despite the different log sensitivity calculations employed in this study, both demonstrate that the log-sensitivity of the fidelity error to dephasing results in a conventional trade-off between performance and robustness.
△ Less
Submitted 16 October, 2023; v1 submitted 9 March, 2023;
originally announced March 2023.
-
Robustness of Energy Landscape Controllers for Spin Rings under Coherent Excitation Transport
Authors:
Sean O'Neil,
Frank Langbein,
Edmond Jonckheere,
S Shermer
Abstract:
The design and analysis of controllers to regulate excitation transport in quantum spin rings presents challenges in the application of classical feedback control techniques to synthesize effective control, and generates results in contradiction to the expectations of classical control theory. In this paper, we examine the robustness of controllers designed to optimize the fidelity of an excitatio…
▽ More
The design and analysis of controllers to regulate excitation transport in quantum spin rings presents challenges in the application of classical feedback control techniques to synthesize effective control, and generates results in contradiction to the expectations of classical control theory. In this paper, we examine the robustness of controllers designed to optimize the fidelity of an excitation transfer to uncertainty in system and control parameters. We use the logarithmic sensitivity of the fidelity error as the measure of robustness, drawing on the classical control analog of the sensitivity of the tracking error. In our analysis we demonstrate that quantum systems optimized for coherent transport demonstrate significantly different correlation between error and the log-sensitivity depending on whether the controller is optimized for readout at an exact time T or over a time-window about T.
△ Less
Submitted 20 August, 2023; v1 submitted 28 February, 2023;
originally announced March 2023.
-
Statistically Characterising Robustness and Fidelity of Quantum Controls and Quantum Control Algorithms
Authors:
Irtaza Khalid,
Carrie A. Weidner,
Edmond A. Jonckheere,
Sophie G. Shermer,
Frank C. Langbein
Abstract:
Robustness of quantum operations or controls is important to build reliable quantum devices. The robustness-infidelity measure (RIM$_p$) is introduced to statistically quantify the robustness and fidelity of a controller as the p-order Wasserstein distance between the fidelity distribution of the controller under any uncertainty and an ideal fidelity distribution. The RIM$_p$ is the p-th root of t…
▽ More
Robustness of quantum operations or controls is important to build reliable quantum devices. The robustness-infidelity measure (RIM$_p$) is introduced to statistically quantify the robustness and fidelity of a controller as the p-order Wasserstein distance between the fidelity distribution of the controller under any uncertainty and an ideal fidelity distribution. The RIM$_p$ is the p-th root of the p-th raw moment of the infidelity distribution. Using a metrization argument, we justify why RIM$_1$ (the average infidelity) suffices as a practical robustness measure. Based on the RIM$_p$, an algorithmic robustness-infidelity measure (ARIM) is developed to quantify the expected robustness and fidelity of controllers found by a control algorithm. The utility of the RIM and ARIM is demonstrated by considering the problem of robust control of spin-$\tfrac{1}{2}$ networks using energy landscape sha** subject to Hamiltonian uncertainty. The robustness and fidelity of individual control solutions as well as the expected robustness and fidelity of controllers found by different popular quantum control algorithms are characterized. For algorithm comparisons, stochastic and non-stochastic optimization objectives are considered, with the goal of effective RIM optimization in the latter. Although high fidelity and robustness are often conflicting objectives, some high fidelity, robust controllers can usually be found, irrespective of the choice of the quantum control algorithm. However, for noisy optimization objectives, adaptive sequential decision making approaches such as reinforcement learning have a cost advantage compared to standard control algorithms and, in contrast, the infidelities obtained are more consistent with higher RIM values for low noise levels.
△ Less
Submitted 1 March, 2023; v1 submitted 15 July, 2022;
originally announced July 2022.
-
Comparison of R1 Map** Protocols: What are we measuring?
Authors:
Christopher Jenkins,
Ioannis Papadopoulos,
S. M. Shermer
Abstract:
Purpose: Recent work highlights the breadth of reported spin-lattice relaxation rates ($R_1$) for individual tissues. One potential source of variation is the protocol used to determine $R_1$. The methodological dependence of R1 and relaxivity $r_1$ are investigated.
Methods: $R_1$ is quantified in gel phantoms with varying concentration of MnCl2, and a small cohort of three healthy volunteers u…
▽ More
Purpose: Recent work highlights the breadth of reported spin-lattice relaxation rates ($R_1$) for individual tissues. One potential source of variation is the protocol used to determine $R_1$. The methodological dependence of R1 and relaxivity $r_1$ are investigated.
Methods: $R_1$ is quantified in gel phantoms with varying concentration of MnCl2, and a small cohort of three healthy volunteers using different acquisition methods. Siemens inversion recovery (IR) and saturation recovery (SR) protocols are applied to phantoms and volunteers. Variable flip angle (VFA) protocols are additionally applied to phantoms. $R_1$ is quantified using single voxel fits, and distributions examined for regions in the thalamus, and cerebellum as well as grey and white matter. Phantoms exclude boundary fits and relaxivity is quantified across the full concentration range. Normality of $R_1$ distributions is assessed by Kolmogorov-Smirnov score, and inter-sequence agreement by two-sample t-test.
Results: Phantom relaxivity is found to be 7.16 Hz/mM, 9.22 Hz/mM and 10.65 Hz/mM to 11.91 Hz/mM for IR, SR and VFA methods, respectively. In vivo $R_1$ exhibit low intra-participant variation for IR. SR $R_1$ are lower than IR values with inter- and intra-participant variation on the same order. Brain regions and phantoms mapped with different protocols varied significantly with t-test p-values between 0 and 5E-10.
Conclusion: Results suggest a significant protocol dependence of $R_1$, and corresponding relaxivity, suggesting inter-method comparisons should be attempted tentatively, if at all.
△ Less
Submitted 27 September, 2019;
originally announced September 2019.
-
MRSNet: Metabolite Quantification from Edited Magnetic Resonance Spectra With Convolutional Neural Networks
Authors:
M. Chandler,
C. Jenkins,
S. M. Shermer,
F. C. Langbein
Abstract:
Quantification of metabolites from magnetic resonance spectra (MRS) has many applications in medicine and psychology, but remains a challenging task despite considerable research efforts. For example, the neurotransmitter $γ$-aminobutyric acid (GABA), present in very low concentration in vivo, regulates inhibitory neurotransmission in the brain and is involved in several processes outside the brai…
▽ More
Quantification of metabolites from magnetic resonance spectra (MRS) has many applications in medicine and psychology, but remains a challenging task despite considerable research efforts. For example, the neurotransmitter $γ$-aminobutyric acid (GABA), present in very low concentration in vivo, regulates inhibitory neurotransmission in the brain and is involved in several processes outside the brain. Reliable quantification is required to determine its role in various physiological and pathological conditions. We present a novel approach to quantification of metabolites from MRS with convolutional neural networks --- MRSNet. MRSNet is trained to perform the multi-class regression problem of identifying relative metabolite concentrations from given input spectra, focusing specifically on the quantification of GABA, which is particularly difficult to resolve. Typically it can only be detected at all using special editing acquisition sequences such as MEGA-PRESS. A large range of network structures, data representations and automatic processing methods are investigated. Results are benchmarked using experimental datasets from test objects of known composition and compared to state-of-the-art quantification methods: LCModel, jMRUI (AQUES, QUEST), TARQUIN, VeSPA and Gannet. The results show that the overall accuracy and precision of metabolite quantification is improved using convolutional neural networks.
△ Less
Submitted 6 September, 2019;
originally announced September 2019.
-
Benchmarking GABA Quantification: A Ground Truth Data Set and Comparative Analysis of TARQUIN, LCModel, jMRUI and Gannet
Authors:
C. Jenkins,
M. Chandler,
F. C. Langbein,
S. M. Shermer
Abstract:
Many tools exist for the quantification of GABA-edited magnetic resonance spectroscopy (MRS) data. Despite a recent consensus effort by the MRS community, literature comparing them is sparse but indicates a methodological bias. While invivo data sets can ascertain the level of agreement between tools, ground-truth is required to establish accuracy, and investigate the sources of discrepancy. We pr…
▽ More
Many tools exist for the quantification of GABA-edited magnetic resonance spectroscopy (MRS) data. Despite a recent consensus effort by the MRS community, literature comparing them is sparse but indicates a methodological bias. While invivo data sets can ascertain the level of agreement between tools, ground-truth is required to establish accuracy, and investigate the sources of discrepancy. We present a novel approach to benchmarking GABA quantification tools, using several series of phantom experiments with iterated GABA concentration. Each series presents a different set of background metabolites and environmental conditions allowing comparison of not only individual estimates, but the ability of tools to characterise changes in GABA across a range of potential confounds. The methodology of the phantom experiments is presented, as well as characterisation of the data. We also perform an initial comparative analysis of several common MRS quantification tools and in-house code, to illustrate utility of the dataset and the potential bias introduced by different quantification methods. The GABA-to-NAA ratios reported by each tool are compared to the ground-truth, and estimation accuracy is assessed by linear regression of this relationship. While the linearity of GABA-to-NAA gradients is generally captured by all of the tools, a large variation in the slope, offset and environmental stability of the gradient is observed. The primary driver of differences in linear combination modelling is the choice of basis function. However, tools employing a common basis and pre-processing still produce differences on the order of 4%. Less-strictly parametrised fitting approaches appear to improve the robustness of quantification, but accurate modelling of edit efficiency calculations is still necessary to avoid systematic offsets. In general, the level of variation suggests that comparisons across...
△ Less
Submitted 8 May, 2021; v1 submitted 4 September, 2019;
originally announced September 2019.
-
Design and Characterisation of Tissue-Mimicking Gel Phantoms for Diffusion Kurtosis Imaging
Authors:
Ziyafer Gizem Portakal,
Sophie Shermer,
Emiliano Spezi,
Teresa Perrett,
Nina Tuncel,
Jonathan Phillips
Abstract:
Purpose: The aim of this work was to create tissue-mimicking gel phantoms appropriate for diffusion kurtosis imaging (DKI) for quality assurance, protocol optimization and sequence development.
Methods: A range of agar, agarose and polyvinyl alcohol phantoms with concentrations ranging from 1.0% to 3.5%, 0.5% to 3.0% and 10% to 20%, respectively, and up to 3 g of glass microspheres per 100 ml we…
▽ More
Purpose: The aim of this work was to create tissue-mimicking gel phantoms appropriate for diffusion kurtosis imaging (DKI) for quality assurance, protocol optimization and sequence development.
Methods: A range of agar, agarose and polyvinyl alcohol phantoms with concentrations ranging from 1.0% to 3.5%, 0.5% to 3.0% and 10% to 20%, respectively, and up to 3 g of glass microspheres per 100 ml were created. Diffusion coefficients, excess kurtosis values and relaxation rates were experimentally determined.
Results: The kurtosis values for the plain gels ranged from 0.05 with 95% confidence interval (CI) of $(0.029, 0.071)$ to $0.216(0.185, 0.246)$, well below the kurtosis values reported in the literature for various tissues. The addition of glass microspheres increased the kurtosis of the gels with values up to $0.523(0.465, 0.581)$ observed for gels with the highest concentration of microspheres. Repeat scans of some of the gels after more than six months of storage at room temperature indicate changes in the diffusion parameters of less than 10%. The addition of the glass microspheres reduces the apparent diffusion coefficients (ADCs) and increases the longitudinal and transverse relaxation rates but the values remain comparable to those for plain gels and tissue, with ADCs observed ranging from $818(585, 1053) \times 10^{-6}$ mm$^2$/s to $2257(2118, 2296) \times 10^{-6}$ mm$^2$/s, and R1 values ranging from $0.34(0.32, 0.35)$ 1/s to $0.51(0.50, 0.52)$ 1/s, and R2 values ranging from $9.69(9.34, 10.04)$ 1/s to $33.07(27.10, 39.04)$ 1/s.
Conclusions: Glass microspheres can be used to effectively modify diffusion properties of gel phantoms and achieve a range of kurtosis values comparable to those reported for a variety of tissues.
△ Less
Submitted 8 March, 2018; v1 submitted 30 August, 2016;
originally announced August 2016.
-
Twisted CP(N-1) instanton projectors and the N-level quantum density matrix
Authors:
Scott Shermer
Abstract:
Twisted classical solutions to the $\mathbb{C}P^{N-1}$ model play a key role in the analysis of such models on the spatially compactified cylinder $\mathbb{S}_L^1 \times {\mathbb{R}^1}$ and have recently been shown to be important for the resurgent structure of this quantum field theory. Instantons and non-self-dual solutions both fractionalize, and domain walls formed by such topological solution…
▽ More
Twisted classical solutions to the $\mathbb{C}P^{N-1}$ model play a key role in the analysis of such models on the spatially compactified cylinder $\mathbb{S}_L^1 \times {\mathbb{R}^1}$ and have recently been shown to be important for the resurgent structure of this quantum field theory. Instantons and non-self-dual solutions both fractionalize, and domain walls formed by such topological solutions can be associated with $N$-vacua having maximally repulsive energy eigenvalues. The purpose of this paper is to reinforce this view through the investigation of a number of parallels between the $\mathbb{C}P^{N-1}$ model and $N$-level quantum mechanical density matrices. Specifically, we demonstrate the existence of a time-evolution equation for the $\mathbb{C}P^{N-1}$ instanton projector analogous to the Liouville-von Neumann equation in the quantum mechanical formalism. The group theoretical analysis of density matrices and the $\mathbb{C}P^{N-1}$ model are also closely related. Finally, we explore the emergence of geometrical (Berry) phases in both systems and their interrelationship.
△ Less
Submitted 8 October, 2015; v1 submitted 9 December, 2014;
originally announced December 2014.
