Skip to main content

Showing 1–26 of 26 results for author: Kessel, P

.
  1. arXiv:2406.06504  [pdf, other

    cs.LG

    Equivariant Neural Tangent Kernels

    Authors: Philipp Misof, Pan Kessel, Jan E. Gerken

    Abstract: Equivariant neural networks have in recent years become an important technique for guiding architecture selection for neural networks with many applications in domains ranging from medical image analysis to quantum chemistry. In particular, as the most general linear equivariant layers with respect to the regular representation, group convolutions have been highly impactful in numerous application… ▽ More

    Submitted 10 June, 2024; originally announced June 2024.

    Comments: 13 pages + 5 pages appendices

  2. arXiv:2406.06150  [pdf, other

    cs.LG quant-ph

    Physics-Informed Bayesian Optimization of Variational Quantum Circuits

    Authors: Kim A. Nicoli, Christopher J. Anders, Lena Funcke, Tobias Hartung, Karl Jansen, Stefan Kühn, Klaus-Robert Müller, Paolo Stornati, Pan Kessel, Shinichi Nakajima

    Abstract: In this paper, we propose a novel and powerful method to harness Bayesian optimization for Variational Quantum Eigensolvers (VQEs) -- a hybrid quantum-classical protocol used to approximate the ground state of a quantum Hamiltonian. Specifically, we derive a VQE-kernel which incorporates important prior information about quantum circuits: the kernel feature map of the VQE-kernel exactly matches th… ▽ More

    Submitted 10 June, 2024; originally announced June 2024.

    Comments: 36 pages, 17 figures, 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

  3. arXiv:2403.15881  [pdf, other

    cs.LG stat.ML

    Fast and Unified Path Gradient Estimators for Normalizing Flows

    Authors: Lorenz Vaitl, Ludwig Winkler, Lorenz Richter, Pan Kessel

    Abstract: Recent work shows that path gradient estimators for normalizing flows have lower variance compared to standard estimators for variational inference, resulting in improved training. However, they are often prohibitively more expensive from a computational point of view and cannot be applied to maximum likelihood training in a scalable manner, which severely hinders their widespread adoption. In thi… ▽ More

    Submitted 23 March, 2024; originally announced March 2024.

  4. arXiv:2403.03103  [pdf, other

    cs.LG

    Emergent Equivariance in Deep Ensembles

    Authors: Jan E. Gerken, Pan Kessel

    Abstract: We show that deep ensembles become equivariant for all inputs and at all training times by simply using data augmentation. Crucially, equivariance holds off-manifold and for any architecture in the infinite width limit. The equivariance is emergent in the sense that predictions of individual ensemble members are not equivariant but their collective prediction is. Neural tangent kernel theory is us… ▽ More

    Submitted 15 June, 2024; v1 submitted 5 March, 2024; originally announced March 2024.

    Comments: 11 pages + 17 pages appendices

  5. arXiv:2307.09379  [pdf, other

    stat.ML cs.LG

    Batched Predictors Generalize within Distribution

    Authors: Andreas Loukas, Pan Kessel

    Abstract: We study the generalization properties of batched predictors, i.e., models tasked with predicting the mean label of a small set (or batch) of examples. The batched prediction paradigm is particularly relevant for models deployed to determine the quality of a group of compounds in preparation for offline testing. By utilizing a suitable generalization of the Rademacher complexity, we prove that bat… ▽ More

    Submitted 18 July, 2023; originally announced July 2023.

    Comments: 9 pages, 3 figures

  6. arXiv:2302.14082  [pdf, other

    hep-lat cs.LG physics.comp-ph

    Detecting and Mitigating Mode-Collapse for Flow-based Sampling of Lattice Field Theories

    Authors: Kim A. Nicoli, Christopher J. Anders, Tobias Hartung, Karl Jansen, Pan Kessel, Shinichi Nakajima

    Abstract: We study the consequences of mode-collapse of normalizing flows in the context of lattice field theory. Normalizing flows allow for independent sampling. For this reason, it is hoped that they can avoid the tunneling problem of local-update MCMC algorithms for multi-modal distributions. In this work, we first point out that the tunneling problem is also present for normalizing flows but is shifted… ▽ More

    Submitted 3 November, 2023; v1 submitted 27 February, 2023; originally announced February 2023.

    Comments: 10 pages, 7 figures, 6 pages of supplement material

  7. Learning Trivializing Gradient Flows for Lattice Gauge Theories

    Authors: Simone Bacchio, Pan Kessel, Stefan Schaefer, Lorenz Vaitl

    Abstract: We propose a unifying approach that starts from the perturbative construction of trivializing maps by Lüscher and then improves on it by learning. The resulting continuous normalizing flow model can be implemented using common tools of lattice field theory and requires several orders of magnitude fewer parameters than any existing machine learning approach. Specifically, our model can achieve comp… ▽ More

    Submitted 9 March, 2023; v1 submitted 16 December, 2022; originally announced December 2022.

    Comments: 10 pages, 4 figures, 1 table

  8. arXiv:2207.08219  [pdf, other

    cs.LG stat.ML

    Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flows

    Authors: Lorenz Vaitl, Kim A. Nicoli, Shinichi Nakajima, Pan Kessel

    Abstract: We propose an algorithm to estimate the path-gradient of both the reverse and forward Kullback-Leibler divergence for an arbitrary manifestly invertible normalizing flow. The resulting path-gradient estimators are straightforward to implement, have lower variance, and lead not only to faster convergence of training but also to better overall approximation results compared to standard total gradien… ▽ More

    Submitted 17 July, 2022; originally announced July 2022.

    Comments: 29 pages, 8 figures

  9. arXiv:2206.09016  [pdf, other

    cs.LG stat.ML

    Path-Gradient Estimators for Continuous Normalizing Flows

    Authors: Lorenz Vaitl, Kim A. Nicoli, Shinichi Nakajima, Pan Kessel

    Abstract: Recent work has established a path-gradient estimator for simple variational Gaussian distributions and has argued that the path-gradient is particularly beneficial in the regime in which the variational distribution approaches the exact target distribution. In many applications, this regime can however not be reached by a simple Gaussian variational distribution. In this work, we overcome this cr… ▽ More

    Submitted 17 June, 2022; originally announced June 2022.

    Comments: 8 pages, 5 figures, 39th International Conference on Machine Learning

  10. arXiv:2206.05075  [pdf, other

    cs.LG cs.AI

    Diffeomorphic Counterfactuals with Generative Models

    Authors: Ann-Kathrin Dombrowski, Jan E. Gerken, Klaus-Robert Müller, Pan Kessel

    Abstract: Counterfactuals can explain classification decisions of neural networks in a human interpretable way. We propose a simple but effective method to generate such counterfactuals. More specifically, we perform a suitable diffeomorphic coordinate transformation and then perform gradient ascent in these coordinates to find counterfactuals which are classified with great confidence as a specified target… ▽ More

    Submitted 16 June, 2022; v1 submitted 10 June, 2022; originally announced June 2022.

  11. arXiv:2111.11303  [pdf, ps, other

    hep-lat cs.LG

    Machine Learning of Thermodynamic Observables in the Presence of Mode Collapse

    Authors: Kim A. Nicoli, Christopher Anders, Lena Funcke, Tobias Hartung, Karl Jansen, Pan Kessel, Shinichi Nakajima, Paolo Stornati

    Abstract: Estimating the free energy, as well as other thermodynamic observables, is a key task in lattice field theories. Recently, it has been pointed out that deep generative models can be used in this context [1]. Crucially, these models allow for the direct estimation of the free energy at a given point in parameter space. This is in contrast to existing methods based on Markov chains which generically… ▽ More

    Submitted 30 November, 2021; v1 submitted 22 November, 2021; originally announced November 2021.

    Comments: 10 pages, 2 figures, Proceedings of the 38th International Symposium on Lattice Field Theory, 26th-30th July 2021, Zoom/Gather@Massachusetts Institute of Technology

    Report number: MIT-CTP/5353

  12. arXiv:2108.10105  [pdf, other

    astro-ph.EP cs.LG physics.flu-dyn physics.geo-ph

    Deep learning for surrogate modelling of 2D mantle convection

    Authors: Siddhant Agarwal, Nicola Tosi, Pan Kessel, Doris Breuer, Grégoire Montavon

    Abstract: Traditionally, 1D models based on scaling laws have been used to parameterized convective heat transfer rocks in the interior of terrestrial planets like Earth, Mars, Mercury and Venus to tackle the computational bottleneck of high-fidelity forward runs in 2D or 3D. However, these are limited in the amount of physics they can model (e.g. depth dependent material properties) and predict only mean q… ▽ More

    Submitted 5 November, 2021; v1 submitted 23 August, 2021; originally announced August 2021.

    Journal ref: Physical Review Fluids, vol. 6, no. 11, 2021

  13. arXiv:2012.10425  [pdf, other

    cs.LG

    Towards Robust Explanations for Deep Neural Networks

    Authors: Ann-Kathrin Dombrowski, Christopher J. Anders, Klaus-Robert Müller, Pan Kessel

    Abstract: Explanation methods shed light on the decision process of black-box classifiers such as deep neural networks. But their usefulness can be compromised because they are susceptible to manipulations. With this work, we aim to enhance the resilience of explanations. We develop a unified theoretical framework for deriving bounds on the maximal manipulability of a model. Based on these theoretical insig… ▽ More

    Submitted 18 December, 2020; originally announced December 2020.

  14. arXiv:2007.09969  [pdf, other

    cs.LG stat.ML

    Fairwashing Explanations with Off-Manifold Detergent

    Authors: Christopher J. Anders, Plamen Pasliev, Ann-Kathrin Dombrowski, Klaus-Robert Müller, Pan Kessel

    Abstract: Explanation methods promise to make black-box classifiers more transparent. As a result, it is hoped that they can act as proof for a sensible, fair and trustworthy decision-making process of the algorithm and thereby increase its acceptance by the end-users. In this paper, we show both theoretically and experimentally that these hopes are presently unfounded. Specifically, we show that, for any c… ▽ More

    Submitted 20 July, 2020; originally announced July 2020.

    Comments: 22 pages with 43 figures, to be published in ICML2020

  15. arXiv:2007.07115  [pdf, other

    hep-lat cs.LG physics.comp-ph

    Estimation of Thermodynamic Observables in Lattice Field Theories with Deep Generative Models

    Authors: Kim A. Nicoli, Christopher J. Anders, Lena Funcke, Tobias Hartung, Karl Jansen, Pan Kessel, Shinichi Nakajima, Paolo Stornati

    Abstract: In this work, we demonstrate that applying deep generative machine learning models for lattice field theory is a promising route for solving problems where Markov Chain Monte Carlo (MCMC) methods are problematic. More specifically, we show that generative models can be used to estimate the absolute value of the free energy, which is in contrast to existing MCMC-based methods which are limited to o… ▽ More

    Submitted 5 January, 2021; v1 submitted 14 July, 2020; originally announced July 2020.

    Comments: 8 figures

    Journal ref: Phys. Rev. Lett. 126, 032001 (2021)

  16. arXiv:1910.13496  [pdf, other

    cond-mat.stat-mech cs.LG stat.ML

    Asymptotically unbiased estimation of physical observables with neural samplers

    Authors: Kim A. Nicoli, Shinichi Nakajima, Nils Strodthoff, Wojciech Samek, Klaus-Robert Müller, Pan Kessel

    Abstract: We propose a general framework for the estimation of observables with generative neural samplers focusing on modern deep generative neural networks that provide an exact sampling probability. In this framework, we present asymptotically unbiased estimators for generic observables, including those that explicitly depend on the partition function such as free energy or entropy, and derive correspond… ▽ More

    Submitted 13 February, 2020; v1 submitted 29 October, 2019; originally announced October 2019.

    Comments: 5 figures

    Journal ref: Phys. Rev. E 101, 023304 (2020)

  17. arXiv:1906.07983  [pdf, other

    stat.ML cs.CR cs.LG

    Explanations can be manipulated and geometry is to blame

    Authors: Ann-Kathrin Dombrowski, Maximilian Alber, Christopher J. Anders, Marcel Ackermann, Klaus-Robert Müller, Pan Kessel

    Abstract: Explanation methods aim to make neural networks more trustworthy and interpretable. In this paper, we demonstrate a property of explanation methods which is disconcerting for both of these purposes. Namely, we show that explanations can be manipulated arbitrarily by applying visually hardly perceptible perturbations to the input that keep the network's output approximately constant. We establish t… ▽ More

    Submitted 25 September, 2019; v1 submitted 19 June, 2019; originally announced June 2019.

  18. arXiv:1903.11048  [pdf, other

    cond-mat.stat-mech cs.LG stat.ML

    Comment on "Solving Statistical Mechanics Using VANs": Introducing saVANt - VANs Enhanced by Importance and MCMC Sampling

    Authors: Kim Nicoli, Pan Kessel, Nils Strodthoff, Wojciech Samek, Klaus-Robert Müller, Shinichi Nakajima

    Abstract: In this comment on "Solving Statistical Mechanics Using Variational Autoregressive Networks" by Wu et al., we propose a subtle yet powerful modification of their approach. We show that the inherent sampling error of their method can be corrected by using neural network-based MCMC or importance sampling which leads to asymptotically unbiased estimators for physical quantities. This modification is… ▽ More

    Submitted 26 March, 2019; originally announced March 2019.

    Comments: 6 pages, 4 figures

  19. arXiv:1810.09751  [pdf, other

    physics.comp-ph physics.chem-ph stat.ML

    Analysis of Atomistic Representations Using Weighted Skip-Connections

    Authors: Kim A. Nicoli, Pan Kessel, Michael Gastegger, Kristof T. Schütt

    Abstract: In this work, we extend the SchNet architecture by using weighted skip connections to assemble the final representation. This enables us to study the relative importance of each interaction block for property prediction. We demonstrate on both the QM9 and MD17 dataset that their relative weighting depends strongly on the chemical composition and configurational degrees of freedom of the molecules… ▽ More

    Submitted 14 November, 2018; v1 submitted 23 October, 2018; originally announced October 2018.

    Comments: NIPS 2018 Workshop: Machine Learning for Molecules and Materials

  20. arXiv:1809.01072  [pdf, other

    physics.comp-ph physics.chem-ph

    SchNetPack: A Deep Learning Toolbox For Atomistic Systems

    Authors: K. T. Schütt, P. Kessel, M. Gastegger, K. Nicoli, A. Tkatchenko, K. -R. Müller

    Abstract: SchNetPack is a toolbox for the development and application of deep neural networks to the prediction of potential energy surfaces and other quantum-chemical properties of molecules and materials. It contains basic building blocks of atomistic neural networks, manages their training and provides simple access to common benchmark datasets. This allows for an easy implementation and evaluation of ne… ▽ More

    Submitted 4 September, 2018; originally announced September 2018.

  21. Simple Unfolded Equations for Massive Higher Spins in AdS$_3$

    Authors: Pan Kessel, Joris Raeymaekers

    Abstract: We propose a simple unfolded description of free massive higher spin particles in anti-de-Sitter spacetime. While our unfolded equation of motion has the standard form of a covariant constancy condition, our formulation differs from the standard one in that our field takes values in a different internal space, which for us is simply a unitary irreducible representation of the symmetry group. Our m… ▽ More

    Submitted 20 August, 2018; v1 submitted 18 May, 2018; originally announced May 2018.

    Comments: 21 pages plus appendices. V2: typos corrected, references added, published version

    Journal ref: JHEP 1808 (2018) 076

  22. Cubic interactions of massless bosonic fields in three dimensions II: Parity-odd and Chern-Simons vertices

    Authors: Pan Kessel, Karapet Mkrtchyan

    Abstract: This work completes the classification of the cubic vertices for arbitrary spin massless bosons in three dimensions started in a previous companion paper by constructing parity-odd vertices. Similarly to the parity-even case, there is a unique parity-odd vertex for any given triple $s_1\geq s_2\geq s_3\geq 2$ of massless bosons if the triangle inequalities are satisfied ($s_1<s_2+s_3$) and none ot… ▽ More

    Submitted 7 March, 2018; originally announced March 2018.

    Comments: 29 pages

    Journal ref: Phys. Rev. D 97, 106021 (2018)

  23. arXiv:1702.03694  [pdf, ps, other

    hep-th

    The Very Basics of Higher-Spin Theory

    Authors: Pan Kessel

    Abstract: These notes are based on two lectures given at the Twelfth Modave Summer School in Mathematical Physics 2016. The Fronsdal equation and action for both Minkowski and (A)dS backgrounds are discussed in detail.

    Submitted 13 February, 2017; originally announced February 2017.

    Comments: Contribution to the proceedings of the XII Modave Summer School in Mathematical Physics

  24. Higher Spin Interactions in Four Dimensions: Vasiliev vs. Fronsdal

    Authors: Nicolas Boulanger, Pan Kessel, E. D. Skvortsov, Massimo Taronna

    Abstract: We consider four-dimensional Higher-Spin Theory at the first nontrivial order corresponding to the cubic action. All Higher-Spin interaction vertices are explicitly obtained from Vasiliev's equations. In particular, we obtain the vertices that are not determined solely by the Higher-Spin algebra structure constants. The dictionary between the Fronsdal fields and Higher-Spin connections is found an… ▽ More

    Submitted 12 December, 2015; v1 submitted 17 August, 2015; originally announced August 2015.

    Comments: 56 pages=40+Appendices; 1 figure; typos fixed, one ref added

  25. Higher Spins and Matter Interacting in Dimension Three

    Authors: Pan Kessel, Gustavo Lucena Gomez, E. D. Skvortsov, Massimo Taronna

    Abstract: The spectrum of Prokushkin--Vasiliev Theory is puzzling in light of the Gaberdiel--Gopakumar conjecture because it generically contains an additional sector besides higher-spin gauge and scalar fields. We find the unique truncation of the theory avoiding this problem to order 2 in perturbations around AdS$_3$. The second-order backreaction on the physical gauge sector induced by the scalars is com… ▽ More

    Submitted 17 November, 2015; v1 submitted 21 May, 2015; originally announced May 2015.

    Comments: 55 pages + appendices, LaTex. Final version to appear in JHEP

  26. Metric- and frame-like higher-spin gauge theories in three dimensions

    Authors: Stefan Fredenhagen, Pan Kessel

    Abstract: We study the relation between the frame-like and metric-like formulation of higher-spin gauge theories in three space-time dimensions. We concentrate on the theory that is described by an SL(3) x SL(3) Chern-Simons theory in the frame-like formulation. The metric-like theory is obtained by eliminating the generalised spin connection by its equation of motion, and by expressing everything in terms… ▽ More

    Submitted 12 August, 2014; originally announced August 2014.

    Comments: 26 pages, no figures