-
Times Square sampling: an adaptive algorithm for free energy estimation
Authors:
Cristian Predescu,
Michael Snarski,
Avi Robinson-Mosher,
Duluxan Sritharan,
Tamas Szalay,
David E. Shaw
Abstract:
Estimating free energy differences, an important problem in computational drug discovery and in a wide range of other application areas, commonly involves a computationally intensive process of sampling a family of high-dimensional probability distributions and a procedure for computing estimates based on those samples. The variance of the free energy estimate of interest typically depends strongl…
▽ More
Estimating free energy differences, an important problem in computational drug discovery and in a wide range of other application areas, commonly involves a computationally intensive process of sampling a family of high-dimensional probability distributions and a procedure for computing estimates based on those samples. The variance of the free energy estimate of interest typically depends strongly on how the total computational resources available for sampling are divided among the distributions, but determining an efficient allocation is difficult without sampling the distributions. Here we introduce the Times Square sampling algorithm, a novel on-the-fly estimation method that dynamically allocates resources in such a way as to significantly accelerate the estimation of free energies and other observables, while providing rigorous convergence guarantees for the estimators. We also show that it is possible, surprisingly, for on-the-fly free energy estimation to achieve lower asymptotic variance than the maximum-likelihood estimator MBAR, raising the prospect that on-the-fly estimation could reduce variance in a variety of other statistical applications.
△ Less
Submitted 9 December, 2021;
originally announced December 2021.
-
Hamilton-Jacobi-Bellman equations for Chemical Reaction Networks
Authors:
Michael Snarski
Abstract:
This is an expository note on large deviations, Hamilton-Jacobi-Bellman (HJB) equations, and the role of the Freidlin-Wentzell quasipotential in Chemical Reaction Networks (CRNs). The note was motivated by observations which identified Lyapunov functions for deterministic descriptions of CRNs by taking appropriate scaling limits of invariant distributions for the corresponding stochastic dynamics.…
▽ More
This is an expository note on large deviations, Hamilton-Jacobi-Bellman (HJB) equations, and the role of the Freidlin-Wentzell quasipotential in Chemical Reaction Networks (CRNs). The note was motivated by observations which identified Lyapunov functions for deterministic descriptions of CRNs by taking appropriate scaling limits of invariant distributions for the corresponding stochastic dynamics. We explain how this is a special case of a classical theory due to Freidlin and Wentzell. We also show that this Lyapunov function is a solution to the HJB partial differential equation if and only if the network is "complex-balanced". The target audience are researchers in the CRN community who are familiar with the Markov process description of CRNs, but do not have the time to invest in learning the technical machinery of large deviations. We conclude by exploring some possible relationships with Kähler geometry which suggest an interesting and unexplored research direction.
△ Less
Submitted 11 August, 2021; v1 submitted 1 August, 2021;
originally announced August 2021.
-
Efficient hyperparameter optimization by way of PAC-Bayes bound minimization
Authors:
John J. Cherian,
Andrew G. Taube,
Robert T. McGibbon,
Panagiotis Angelikopoulos,
Guy Blanc,
Michael Snarski,
Daniel D. Richman,
John L. Klepeis,
David E. Shaw
Abstract:
Identifying optimal values for a high-dimensional set of hyperparameters is a problem that has received growing attention given its importance to large-scale machine learning applications such as neural architecture search. Recently developed optimization methods can be used to select thousands or even millions of hyperparameters. Such methods often yield overfit models, however, leading to poor p…
▽ More
Identifying optimal values for a high-dimensional set of hyperparameters is a problem that has received growing attention given its importance to large-scale machine learning applications such as neural architecture search. Recently developed optimization methods can be used to select thousands or even millions of hyperparameters. Such methods often yield overfit models, however, leading to poor performance on unseen data. We argue that this overfitting results from using the standard hyperparameter optimization objective function. Here we present an alternative objective that is equivalent to a Probably Approximately Correct-Bayes (PAC-Bayes) bound on the expected out-of-sample error. We then devise an efficient gradient-based algorithm to minimize this objective; the proposed method has asymptotic space and time complexity equal to or better than other gradient-based hyperparameter optimization methods. We show that this new method significantly reduces out-of-sample error when applied to hyperparameter optimization problems known to be prone to overfitting.
△ Less
Submitted 14 August, 2020;
originally announced August 2020.
-
Infinite Swap** using IID Samples
Authors:
Paul Dupuis,
Guo-Jhen Wu,
Michael Snarski
Abstract:
We propose a new method for estimating rare event probabilities when independent samples are available. It is assumed that the underlying probability measures satisfy a large deviations principle with a scaling parameter $\varepsilon$ that we call temperature. We show how by combining samples at different temperatures, one can construct an estimator with greatly reduced variance. Although as prese…
▽ More
We propose a new method for estimating rare event probabilities when independent samples are available. It is assumed that the underlying probability measures satisfy a large deviations principle with a scaling parameter $\varepsilon$ that we call temperature. We show how by combining samples at different temperatures, one can construct an estimator with greatly reduced variance. Although as presented here the method is not as broadly applicable as other rare event simulation methods, such as splitting or importance sampling, it does not require any problem-dependent constructions.
△ Less
Submitted 4 November, 2018;
originally announced November 2018.
-
Periodic Solutions of a Singularly Perturbed Delay Differential Equation With Two State-Dependent Delays
Authors:
A. R. Humphries,
D. A. Bernucci,
R. Calleja,
N. Homayounfar,
M. Snarski
Abstract:
Periodic orbits and associated bifurcations of singularly perturbed state-dependent delay differential equations (DDEs) are studied when the profiles of the periodic orbits contain jump discontinuities in the singular limit. A definition of singular solution is introduced which is based on a continuous parametrisation of the possibly discontinuous limiting solution. This reduces the construction o…
▽ More
Periodic orbits and associated bifurcations of singularly perturbed state-dependent delay differential equations (DDEs) are studied when the profiles of the periodic orbits contain jump discontinuities in the singular limit. A definition of singular solution is introduced which is based on a continuous parametrisation of the possibly discontinuous limiting solution. This reduces the construction of the limiting profiles to an algebraic problem. A model two state-dependent delay differential equation is studied in detail and periodic singular solutions are constructed with one and two local maxima per period. A complete characterisation of the conditions on the parameters for these singular solutions to exist facilitates an investigation of bifurcation structures in the singular case revealing folds and possible cusp bifurcations. Sophisticated boundary value techniques are used to numerically compute the bifurcation diagram of the state-dependent DDE when the perturbation parameter is close to zero. This confirms that the solutions and bifurcations constructed in the singular case persist when the perturbation parameter is nonzero, and hence demonstrates that the solutions constructed using our singular solution definition are useful and relevant to the singularly perturbed problem. Fold and cusp bifurcations are found very close to the parameter values predicted by the singular solution theory, and we also find period-doubling bifurcations as well as periodic orbits with more than two local maxima per period, and explain the alignment between the folds on different bifurcation branches.
△ Less
Submitted 9 June, 2015; v1 submitted 21 November, 2014;
originally announced November 2014.
-
Analysis of a variational model for nematic shells
Authors:
Antonio Segatti,
Michael Snarski,
Marco Veneroni
Abstract:
We analyze an elastic surface energy which was recently introduced by G. Napoli and L.Vergori to model thin films of nematic liquid crystals. We show how a novel approach that takes into account also the extrinsic properties of the surfaces coated by the liquid crystal leads to considerable differences with respect to the classical intrinsic energy. Our results concern three connected aspects: i)…
▽ More
We analyze an elastic surface energy which was recently introduced by G. Napoli and L.Vergori to model thin films of nematic liquid crystals. We show how a novel approach that takes into account also the extrinsic properties of the surfaces coated by the liquid crystal leads to considerable differences with respect to the classical intrinsic energy. Our results concern three connected aspects: i) using methods of the calculus of variations, we establish a relation between the existence of minimizers and the topology of the surface; ii) we prove, by a Ginzburg-Landau approximation, the well-posedness of the gradient flow of the energy; iii) in the case of a parametrized axisymmetric torus we obtain a stronger characterization of global and local minimizers, which we supplement with numerical experiments.
△ Less
Submitted 21 June, 2016; v1 submitted 12 August, 2014;
originally announced August 2014.
-
Equilibrium configurations of nematic liquid crystals on a torus
Authors:
Antonio Segatti,
Michael Snarski,
Marco Veneroni
Abstract:
The topology and the geometry of a surface play a fundamental role in determining the equilibrium configurations of thin films of liquid crystals. We propose here a theoretical analysis of a recently introduced surface Frank energy, in the case of two-dimensional nematic liquid crystals coating a toroidal particle. Our aim is to show how a different modeling of the effect of extrinsic curvature ac…
▽ More
The topology and the geometry of a surface play a fundamental role in determining the equilibrium configurations of thin films of liquid crystals. We propose here a theoretical analysis of a recently introduced surface Frank energy, in the case of two-dimensional nematic liquid crystals coating a toroidal particle. Our aim is to show how a different modeling of the effect of extrinsic curvature acts as a selection principle among equilibria of the classical energy, and how new configurations emerge. In particular, our analysis predicts the existence of new stable equilibria with complex windings.
△ Less
Submitted 13 June, 2014; v1 submitted 23 January, 2014;
originally announced January 2014.
