-
Practical applications of machine-learned flows on gauge fields
Authors:
Ryan Abbott,
Michael S. Albergo,
Denis Boyda,
Daniel C. Hackett,
Gurtej Kanwar,
Fernando Romero-López,
Phiala E. Shanahan,
Julian M. Urban
Abstract:
Normalizing flows are machine-learned maps between different lattice theories which can be used as components in exact sampling and inference schemes. Ongoing work yields increasingly expressive flows on gauge fields, but it remains an open question how flows can improve lattice QCD at state-of-the-art scales. We discuss and demonstrate two applications of flows in replica exchange (parallel tempe…
▽ More
Normalizing flows are machine-learned maps between different lattice theories which can be used as components in exact sampling and inference schemes. Ongoing work yields increasingly expressive flows on gauge fields, but it remains an open question how flows can improve lattice QCD at state-of-the-art scales. We discuss and demonstrate two applications of flows in replica exchange (parallel tempering) sampling, aimed at improving topological mixing, which are viable with iterative improvements upon presently available flows.
△ Less
Submitted 17 April, 2024;
originally announced April 2024.
-
Applications of flow models to the generation of correlated lattice QCD ensembles
Authors:
Ryan Abbott,
Aleksandar Botev,
Denis Boyda,
Daniel C. Hackett,
Gurtej Kanwar,
Sébastien Racanière,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Julian M. Urban
Abstract:
Machine-learned normalizing flows can be used in the context of lattice quantum field theory to generate statistically correlated ensembles of lattice gauge fields at different action parameters. This work demonstrates how these correlations can be exploited for variance reduction in the computation of observables. Three different proof-of-concept applications are demonstrated using a novel residu…
▽ More
Machine-learned normalizing flows can be used in the context of lattice quantum field theory to generate statistically correlated ensembles of lattice gauge fields at different action parameters. This work demonstrates how these correlations can be exploited for variance reduction in the computation of observables. Three different proof-of-concept applications are demonstrated using a novel residual flow architecture: continuum limits of gauge theories, the mass dependence of QCD observables, and hadronic matrix elements based on the Feynman-Hellmann approach. In all three cases, it is shown that statistical uncertainties are significantly reduced when machine-learned flows are incorporated as compared with the same calculations performed with uncorrelated ensembles or direct reweighting.
△ Less
Submitted 28 May, 2024; v1 submitted 19 January, 2024;
originally announced January 2024.
-
Normalizing flows for lattice gauge theory in arbitrary space-time dimension
Authors:
Ryan Abbott,
Michael S. Albergo,
Aleksandar Botev,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Gurtej Kanwar,
Alexander G. D. G. Matthews,
Sébastien Racanière,
Ali Razavi,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Julian M. Urban
Abstract:
Applications of normalizing flows to the sampling of field configurations in lattice gauge theory have so far been explored almost exclusively in two space-time dimensions. We report new algorithmic developments of gauge-equivariant flow architectures facilitating the generalization to higher-dimensional lattice geometries. Specifically, we discuss masked autoregressive transformations with tracta…
▽ More
Applications of normalizing flows to the sampling of field configurations in lattice gauge theory have so far been explored almost exclusively in two space-time dimensions. We report new algorithmic developments of gauge-equivariant flow architectures facilitating the generalization to higher-dimensional lattice geometries. Specifically, we discuss masked autoregressive transformations with tractable and unbiased Jacobian determinants, a key ingredient for scalable and asymptotically exact flow-based sampling algorithms. For concreteness, results from a proof-of-principle application to SU(3) lattice gauge theory in four space-time dimensions are reported.
△ Less
Submitted 3 May, 2023;
originally announced May 2023.
-
Aspects of scaling and scalability for flow-based sampling of lattice QCD
Authors:
Ryan Abbott,
Michael S. Albergo,
Aleksandar Botev,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Alexander G. D. G. Matthews,
Sébastien Racanière,
Ali Razavi,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Julian M. Urban
Abstract:
Recent applications of machine-learned normalizing flows to sampling in lattice field theory suggest that such methods may be able to mitigate critical slowing down and topological freezing. However, these demonstrations have been at the scale of toy models, and it remains to be determined whether they can be applied to state-of-the-art lattice quantum chromodynamics calculations. Assessing the vi…
▽ More
Recent applications of machine-learned normalizing flows to sampling in lattice field theory suggest that such methods may be able to mitigate critical slowing down and topological freezing. However, these demonstrations have been at the scale of toy models, and it remains to be determined whether they can be applied to state-of-the-art lattice quantum chromodynamics calculations. Assessing the viability of sampling algorithms for lattice field theory at scale has traditionally been accomplished using simple cost scaling laws, but as we discuss in this work, their utility is limited for flow-based approaches. We conclude that flow-based approaches to sampling are better thought of as a broad family of algorithms with different scaling properties, and that scalability must be assessed experimentally.
△ Less
Submitted 14 November, 2022;
originally announced November 2022.
-
Neural-network preconditioners for solving the Dirac equation in lattice gauge theory
Authors:
Salvatore Calì,
Daniel C. Hackett,
Yin Lin,
Phiala E. Shanahan,
Brian Xiao
Abstract:
This work develops neural-network--based preconditioners to accelerate solution of the Wilson-Dirac normal equation in lattice quantum field theories. The approach is implemented for the two-flavor lattice Schwinger model near the critical point. In this system, neural-network preconditioners are found to accelerate the convergence of the conjugate gradient solver compared with the solution of unp…
▽ More
This work develops neural-network--based preconditioners to accelerate solution of the Wilson-Dirac normal equation in lattice quantum field theories. The approach is implemented for the two-flavor lattice Schwinger model near the critical point. In this system, neural-network preconditioners are found to accelerate the convergence of the conjugate gradient solver compared with the solution of unpreconditioned systems or those preconditioned with conventional approaches based on even-odd or incomplete Cholesky decompositions, as measured by reductions in the number of iterations and/or complex operations required for convergence. It is also shown that a preconditioner trained on ensembles with small lattice volumes can be used to construct preconditioners for ensembles with many times larger lattice volumes, with minimal degradation of performance. This volume-transferring technique amortizes the training cost and presents a pathway towards scaling such preconditioners to lattice field theory calculations with larger lattice volumes and in four dimensions.
△ Less
Submitted 4 August, 2022;
originally announced August 2022.
-
Data Science and Machine Learning in Education
Authors:
Gabriele Benelli,
Thomas Y. Chen,
Javier Duarte,
Matthew Feickert,
Matthew Graham,
Lindsey Gray,
Dan Hackett,
Phil Harris,
Shih-Chieh Hsu,
Gregor Kasieczka,
Elham E. Khoda,
Matthias Komm,
Mia Liu,
Mark S. Neubauer,
Scarlet Norberg,
Alexx Perloff,
Marcel Rieger,
Claire Savard,
Kazuhiro Terao,
Savannah Thais,
Avik Roy,
Jean-Roch Vlimant,
Grigorios Chachamis
Abstract:
The growing role of data science (DS) and machine learning (ML) in high-energy physics (HEP) is well established and pertinent given the complex detectors, large data, sets and sophisticated analyses at the heart of HEP research. Moreover, exploiting symmetries inherent in physics data have inspired physics-informed ML as a vibrant sub-field of computer science research. HEP researchers benefit gr…
▽ More
The growing role of data science (DS) and machine learning (ML) in high-energy physics (HEP) is well established and pertinent given the complex detectors, large data, sets and sophisticated analyses at the heart of HEP research. Moreover, exploiting symmetries inherent in physics data have inspired physics-informed ML as a vibrant sub-field of computer science research. HEP researchers benefit greatly from materials widely available materials for use in education, training and workforce development. They are also contributing to these materials and providing software to DS/ML-related fields. Increasingly, physics departments are offering courses at the intersection of DS, ML and physics, often using curricula developed by HEP researchers and involving open software and data used in HEP. In this white paper, we explore synergies between HEP research and DS/ML education, discuss opportunities and challenges at this intersection, and propose community activities that will be mutually beneficial.
△ Less
Submitted 19 July, 2022;
originally announced July 2022.
-
Gauge-equivariant flow models for sampling in lattice field theories with pseudofermions
Authors:
Ryan Abbott,
Michael S. Albergo,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Gurtej Kanwar,
Sébastien Racanière,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Betsy Tian,
Julian M. Urban
Abstract:
This work presents gauge-equivariant architectures for flow-based sampling in fermionic lattice field theories using pseudofermions as stochastic estimators for the fermionic determinant. This is the default approach in state-of-the-art lattice field theory calculations, making this development critical to the practical application of flow models to theories such as QCD. Methods by which flow-base…
▽ More
This work presents gauge-equivariant architectures for flow-based sampling in fermionic lattice field theories using pseudofermions as stochastic estimators for the fermionic determinant. This is the default approach in state-of-the-art lattice field theory calculations, making this development critical to the practical application of flow models to theories such as QCD. Methods by which flow-based sampling approaches can be improved via standard techniques such as even/odd preconditioning and the Hasenbusch factorization are also outlined. Numerical demonstrations in two-dimensional U(1) and SU(3) gauge theories with $N_f=2$ flavors of fermions are provided.
△ Less
Submitted 16 October, 2022; v1 submitted 18 July, 2022;
originally announced July 2022.
-
Flow-based sampling in the lattice Schwinger model at criticality
Authors:
Michael S. Albergo,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Gurtej Kanwar,
Sébastien Racanière,
Danilo J. Rezende,
Fernando Romero-López,
Phiala E. Shanahan,
Julian M. Urban
Abstract:
Recent results suggest that flow-based algorithms may provide efficient sampling of field distributions for lattice field theory applications, such as studies of quantum chromodynamics and the Schwinger model. In this work, we provide a numerical demonstration of robust flow-based sampling in the Schwinger model at the critical value of the fermion mass. In contrast, at the same parameters, conven…
▽ More
Recent results suggest that flow-based algorithms may provide efficient sampling of field distributions for lattice field theory applications, such as studies of quantum chromodynamics and the Schwinger model. In this work, we provide a numerical demonstration of robust flow-based sampling in the Schwinger model at the critical value of the fermion mass. In contrast, at the same parameters, conventional methods fail to sample all parts of configuration space, leading to severely underestimated uncertainties.
△ Less
Submitted 23 February, 2022;
originally announced February 2022.
-
Applications of Machine Learning to Lattice Quantum Field Theory
Authors:
Denis Boyda,
Salvatore Calì,
Sam Foreman,
Lena Funcke,
Daniel C. Hackett,
Yin Lin,
Gert Aarts,
Andrei Alexandru,
Xiao-Yong **,
Biagio Lucini,
Phiala E. Shanahan
Abstract:
There is great potential to apply machine learning in the area of numerical lattice quantum field theory, but full exploitation of that potential will require new strategies. In this white paper for the Snowmass community planning process, we discuss the unique requirements of machine learning for lattice quantum field theory research and outline what is needed to enable exploration and deployment…
▽ More
There is great potential to apply machine learning in the area of numerical lattice quantum field theory, but full exploitation of that potential will require new strategies. In this white paper for the Snowmass community planning process, we discuss the unique requirements of machine learning for lattice quantum field theory research and outline what is needed to enable exploration and deployment of this approach in the future.
△ Less
Submitted 10 February, 2022;
originally announced February 2022.
-
Flow-based sampling for multimodal distributions in lattice field theory
Authors:
Daniel C. Hackett,
Chung-Chun Hsieh,
Michael S. Albergo,
Denis Boyda,
Jiunn-Wei Chen,
Kai-Feng Chen,
Kyle Cranmer,
Gurtej Kanwar,
Phiala E. Shanahan
Abstract:
Recent results have demonstrated that samplers constructed with flow-based generative models are a promising new approach for configuration generation in lattice field theory. In this paper, we present a set of methods to construct flow models for targets with multiple separated modes (i.e. theories with multiple vacua). We demonstrate the application of these methods to modeling two-dimensional r…
▽ More
Recent results have demonstrated that samplers constructed with flow-based generative models are a promising new approach for configuration generation in lattice field theory. In this paper, we present a set of methods to construct flow models for targets with multiple separated modes (i.e. theories with multiple vacua). We demonstrate the application of these methods to modeling two-dimensional real scalar field theory in its symmetry-broken phase. In this context we investigate the performance of different flow-based sampling algorithms, including a composite sampling algorithm where flow-based proposals are occasionally augmented by applying updates using traditional algorithms like HMC.
△ Less
Submitted 1 July, 2021;
originally announced July 2021.
-
Flow-based sampling for fermionic lattice field theories
Authors:
Michael S. Albergo,
Gurtej Kanwar,
Sébastien Racanière,
Danilo J. Rezende,
Julian M. Urban,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Phiala E. Shanahan
Abstract:
Algorithms based on normalizing flows are emerging as promising machine learning approaches to sampling complicated probability distributions in a way that can be made asymptotically exact. In the context of lattice field theory, proof-of-principle studies have demonstrated the effectiveness of this approach for scalar theories, gauge theories, and statistical systems. This work develops approache…
▽ More
Algorithms based on normalizing flows are emerging as promising machine learning approaches to sampling complicated probability distributions in a way that can be made asymptotically exact. In the context of lattice field theory, proof-of-principle studies have demonstrated the effectiveness of this approach for scalar theories, gauge theories, and statistical systems. This work develops approaches that enable flow-based sampling of theories with dynamical fermions, which is necessary for the technique to be applied to lattice field theory studies of the Standard Model of particle physics and many condensed matter systems. As a practical demonstration, these methods are applied to the sampling of field configurations for a two-dimensional theory of massless staggered fermions coupled to a scalar field via a Yukawa interaction.
△ Less
Submitted 28 December, 2021; v1 submitted 10 June, 2021;
originally announced June 2021.
-
Introduction to Normalizing Flows for Lattice Field Theory
Authors:
Michael S. Albergo,
Denis Boyda,
Daniel C. Hackett,
Gurtej Kanwar,
Kyle Cranmer,
Sébastien Racanière,
Danilo Jimenez Rezende,
Phiala E. Shanahan
Abstract:
This notebook tutorial demonstrates a method for sampling Boltzmann distributions of lattice field theories using a class of machine learning models known as normalizing flows. The ideas and approaches proposed in arXiv:1904.12072, arXiv:2002.02428, and arXiv:2003.06413 are reviewed and a concrete implementation of the framework is presented. We apply this framework to a lattice scalar field theor…
▽ More
This notebook tutorial demonstrates a method for sampling Boltzmann distributions of lattice field theories using a class of machine learning models known as normalizing flows. The ideas and approaches proposed in arXiv:1904.12072, arXiv:2002.02428, and arXiv:2003.06413 are reviewed and a concrete implementation of the framework is presented. We apply this framework to a lattice scalar field theory and to U(1) gauge theory, explicitly encoding gauge symmetries in the flow-based approach to the latter. This presentation is intended to be interactive and working with the attached Jupyter notebook is recommended.
△ Less
Submitted 6 August, 2021; v1 submitted 20 January, 2021;
originally announced January 2021.
-
Sampling using $SU(N)$ gauge equivariant flows
Authors:
Denis Boyda,
Gurtej Kanwar,
Sébastien Racanière,
Danilo Jimenez Rezende,
Michael S. Albergo,
Kyle Cranmer,
Daniel C. Hackett,
Phiala E. Shanahan
Abstract:
We develop a flow-based sampling algorithm for $SU(N)$ lattice gauge theories that is gauge-invariant by construction. Our key contribution is constructing a class of flows on an $SU(N)$ variable (or on a $U(N)$ variable by a simple alternative) that respect matrix conjugation symmetry. We apply this technique to sample distributions of single $SU(N)$ variables and to construct flow-based samplers…
▽ More
We develop a flow-based sampling algorithm for $SU(N)$ lattice gauge theories that is gauge-invariant by construction. Our key contribution is constructing a class of flows on an $SU(N)$ variable (or on a $U(N)$ variable by a simple alternative) that respect matrix conjugation symmetry. We apply this technique to sample distributions of single $SU(N)$ variables and to construct flow-based samplers for $SU(2)$ and $SU(3)$ lattice gauge theory in two dimensions.
△ Less
Submitted 18 September, 2020; v1 submitted 12 August, 2020;
originally announced August 2020.
-
Equivariant flow-based sampling for lattice gauge theory
Authors:
Gurtej Kanwar,
Michael S. Albergo,
Denis Boyda,
Kyle Cranmer,
Daniel C. Hackett,
Sébastien Racanière,
Danilo Jimenez Rezende,
Phiala E. Shanahan
Abstract:
We define a class of machine-learned flow-based sampling algorithms for lattice gauge theories that are gauge-invariant by construction. We demonstrate the application of this framework to U(1) gauge theory in two spacetime dimensions, and find that near critical points in parameter space the approach is orders of magnitude more efficient at sampling topological quantities than more traditional sa…
▽ More
We define a class of machine-learned flow-based sampling algorithms for lattice gauge theories that are gauge-invariant by construction. We demonstrate the application of this framework to U(1) gauge theory in two spacetime dimensions, and find that near critical points in parameter space the approach is orders of magnitude more efficient at sampling topological quantities than more traditional sampling procedures such as Hybrid Monte Carlo and Heat Bath.
△ Less
Submitted 13 March, 2020;
originally announced March 2020.
