-
Incorporating Memory into Propagation of 1-Electron Reduced Density Matrices
Authors:
Harish S. Bhat,
Hardeep Bassi,
Karnamohit Ranka,
Christine M. Isborn
Abstract:
For any linear system where the unreduced dynamics are governed by unitary propagators, we derive a closed, time-delayed, linear system for a reduced-dimensional quantity of interest. We apply this method to understand the memory-dependence of $1$-electron reduced density matrices in time-dependent configuration interaction (TDCI), a scheme to solve for the correlated dynamics of electrons in mole…
▽ More
For any linear system where the unreduced dynamics are governed by unitary propagators, we derive a closed, time-delayed, linear system for a reduced-dimensional quantity of interest. We apply this method to understand the memory-dependence of $1$-electron reduced density matrices in time-dependent configuration interaction (TDCI), a scheme to solve for the correlated dynamics of electrons in molecules. Though time-dependent density functional theory has established that the $1$-electron reduced density possesses memory-dependence, the precise nature of this memory-dependence has not been understood. We derive a self-contained, symmetry/constraint-preserving method to propagate reduced TDCI electron density matrices. Our method preserves properties of density matrices such as Hermitian symmetry and constant trace. In numerical tests on two model systems ($\text{H}_2$ and $\text{HeH}^+$), we show that with sufficiently large time-delay (or memory-dependence), our method propagates reduced TDCI density matrices with high quantitative accuracy. We study the dependence of our results on time step and basis set. To implement our method, we derive the $4$-index tensor that relates reduced and full TDCI density matrices. Our derivation applies to any TDCI system, regardless of basis set, number of electrons, or choice of Slater determinants in the wave function. This derivation enables a proof that the trace of the reduced TDCI density matrix is constant and equals the number of electrons.
△ Less
Submitted 24 June, 2024; v1 submitted 22 March, 2024;
originally announced March 2024.
-
Neural Continuous-Time Markov Models
Authors:
Majerle Reeves,
Harish S. Bhat
Abstract:
Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a method to learn a continuous-time Markov chain's transition rate functions from fully observed time series. In contrast with existing methods, our method allows for…
▽ More
Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a method to learn a continuous-time Markov chain's transition rate functions from fully observed time series. In contrast with existing methods, our method allows for transition rates to depend nonlinearly on both state variables and external covariates. The Gillespie algorithm is used to generate trajectories of stochastic systems where propensity functions (reaction rates) are known. Our method can be viewed as the inverse: given trajectories of a stochastic reaction network, we generate estimates of the propensity functions. While previous methods used linear or log-linear methods to link transition rates to covariates, we use neural networks, increasing the capacity and potential accuracy of learned models. In the chemical context, this enables the method to learn propensity functions from non-mass-action kinetics. We test our method with synthetic data generated from a variety of systems with known transition rates. We show that our method learns these transition rates with considerably more accuracy than log-linear methods, in terms of mean absolute error between ground truth and predicted transition rates. We also demonstrate an application of our methods to open-loop control of a continuous-time Markov chain.
△ Less
Submitted 10 December, 2022;
originally announced December 2022.
-
Drift Identification for Lévy alpha-Stable Stochastic Systems
Authors:
Harish S. Bhat
Abstract:
This paper focuses on a stochastic system identification problem: given time series observations of a stochastic differential equation (SDE) driven by Lévy $α$-stable noise, estimate the SDE's drift field. For $α$ in the interval $[1,2)$, the noise is heavy-tailed, leading to computational difficulties for methods that compute transition densities and/or likelihoods in physical space. We propose a…
▽ More
This paper focuses on a stochastic system identification problem: given time series observations of a stochastic differential equation (SDE) driven by Lévy $α$-stable noise, estimate the SDE's drift field. For $α$ in the interval $[1,2)$, the noise is heavy-tailed, leading to computational difficulties for methods that compute transition densities and/or likelihoods in physical space. We propose a Fourier space approach that centers on computing time-dependent characteristic functions, i.e., Fourier transforms of time-dependent densities. Parameterizing the unknown drift field using Fourier series, we formulate a loss consisting of the squared error between predicted and empirical characteristic functions. We minimize this loss with gradients computed via the adjoint method. For a variety of one- and two-dimensional problems, we demonstrate that this method is capable of learning drift fields in qualitative and/or quantitative agreement with ground truth fields.
△ Less
Submitted 6 December, 2022;
originally announced December 2022.
-
Dynamic Learning of Correlation Potentials for a Time-Dependent Kohn-Sham System
Authors:
Harish S. Bhat,
Kevin Collins,
Prachi Gupta,
Christine M. Isborn
Abstract:
We develop methods to learn the correlation potential for a time-dependent Kohn-Sham (TDKS) system in one spatial dimension. We start from a low-dimensional two-electron system for which we can numerically solve the time-dependent Schrödinger equation; this yields electron densities suitable for training models of the correlation potential. We frame the learning problem as one of optimizing a leas…
▽ More
We develop methods to learn the correlation potential for a time-dependent Kohn-Sham (TDKS) system in one spatial dimension. We start from a low-dimensional two-electron system for which we can numerically solve the time-dependent Schrödinger equation; this yields electron densities suitable for training models of the correlation potential. We frame the learning problem as one of optimizing a least-squares objective subject to the constraint that the dynamics obey the TDKS equation. Applying adjoints, we develop efficient methods to compute gradients and thereby learn models of the correlation potential. Our results show that it is possible to learn values of the correlation potential such that the resulting electron densities match ground truth densities. We also show how to learn correlation potential functionals with memory, demonstrating one such model that yields reasonable results for trajectories outside the training set.
△ Less
Submitted 6 December, 2022; v1 submitted 13 December, 2021;
originally announced December 2021.
-
Equity-Directed Bootstrap**: Examples and Analysis
Authors:
Harish S. Bhat,
Majerle E. Reeves,
Sidra Goldman-Mellor
Abstract:
When faced with severely imbalanced binary classification problems, we often train models on bootstrapped data in which the number of instances of each class occur in a more favorable ratio, e.g., one. We view algorithmic inequity through the lens of imbalanced classification: in order to balance the performance of a classifier across groups, we can bootstrap to achieve training sets that are bala…
▽ More
When faced with severely imbalanced binary classification problems, we often train models on bootstrapped data in which the number of instances of each class occur in a more favorable ratio, e.g., one. We view algorithmic inequity through the lens of imbalanced classification: in order to balance the performance of a classifier across groups, we can bootstrap to achieve training sets that are balanced with respect to both labels and group identity. For an example problem with severe class imbalance---prediction of suicide death from administrative patient records---we illustrate how an equity-directed bootstrap can bring test set sensitivities and specificities much closer to satisfying the equal odds criterion. In the context of naïve Bayes and logistic regression, we analyze the equity-directed bootstrap, demonstrating that it works by bringing odds ratios close to one, and linking it to methods involving intercept adjustment, thresholding, and weighting.
△ Less
Submitted 14 August, 2021;
originally announced August 2021.
-
Statistical learning method for predicting density-matrix based electron dynamics
Authors:
Prachi Gupta,
Harish S. Bhat,
Karnamohit Ranka,
Christine M. Isborn
Abstract:
We develop a statistical method to learn a molecular Hamiltonian matrix from a time-series of electron density matrices. We extend our previous method to larger molecular systems by incorporating physical properties to reduce dimensionality, while also exploiting regularization techniques like ridge regression for addressing multicollinearity. With the learned Hamiltonian we can solve the Time-Dep…
▽ More
We develop a statistical method to learn a molecular Hamiltonian matrix from a time-series of electron density matrices. We extend our previous method to larger molecular systems by incorporating physical properties to reduce dimensionality, while also exploiting regularization techniques like ridge regression for addressing multicollinearity. With the learned Hamiltonian we can solve the Time-Dependent Hartree-Fock (TDHF) equation to propagate the electron density in time, and predict its dynamics for field-free and field-on scenarios. We observe close quantitative agreement between the predicted dynamics and ground truth for both field-off trajectories similar to the training data, and field-on trajectories outside of the training data.
△ Less
Submitted 31 July, 2021;
originally announced August 2021.
-
Estimating Vector Fields from Noisy Time Series
Authors:
Harish S. Bhat,
Majerle Reeves,
Ramin Raziperchikolaei
Abstract:
While there has been a surge of recent interest in learning differential equation models from time series, methods in this area typically cannot cope with highly noisy data. We break this problem into two parts: (i) approximating the unknown vector field (or right-hand side) of the differential equation, and (ii) dealing with noise. To deal with (i), we describe a neural network architecture consi…
▽ More
While there has been a surge of recent interest in learning differential equation models from time series, methods in this area typically cannot cope with highly noisy data. We break this problem into two parts: (i) approximating the unknown vector field (or right-hand side) of the differential equation, and (ii) dealing with noise. To deal with (i), we describe a neural network architecture consisting of tensor products of one-dimensional neural shape functions. For (ii), we propose an alternating minimization scheme that switches between vector field training and filtering steps, together with multiple trajectories of training data. We find that the neural shape function architecture retains the approximation properties of dense neural networks, enables effective computation of vector field error, and allows for graphical interpretability, all for data/systems in any finite dimension $d$. We also study the combination of either our neural shape function method or existing differential equation learning methods with alternating minimization and multiple trajectories. We find that retrofitting any learning method in this way boosts the method's robustness to noise. While in their raw form the methods struggle with 1% Gaussian noise, after retrofitting, they learn accurate vector fields from data with 10% Gaussian noise.
△ Less
Submitted 6 December, 2020;
originally announced December 2020.
-
A non-autonomous equation discovery method for time signal classification
Authors:
Ryeongkyung Yoon,
Harish S. Bhat,
Braxton Osting
Abstract:
Certain neural network architectures, in the infinite-layer limit, lead to systems of nonlinear differential equations. Motivated by this idea, we develop a framework for analyzing time signals based on non-autonomous dynamical equations. We view the time signal as a forcing function for a dynamical system that governs a time-evolving hidden variable. As in equation discovery, the dynamical system…
▽ More
Certain neural network architectures, in the infinite-layer limit, lead to systems of nonlinear differential equations. Motivated by this idea, we develop a framework for analyzing time signals based on non-autonomous dynamical equations. We view the time signal as a forcing function for a dynamical system that governs a time-evolving hidden variable. As in equation discovery, the dynamical system is represented using a dictionary of functions and the coefficients are learned from data. This framework is applied to the time signal classification problem. We show how gradients can be efficiently computed using the adjoint method, and we apply methods from dynamical systems to establish stability of the classifier. Through a variety of experiments, on both synthetic and real datasets, we show that the proposed method uses orders of magnitude fewer parameters than competing methods, while achieving comparable accuracy. We created the synthetic datasets using dynamical systems of increasing complexity; though the ground truth vector fields are often polynomials, we find consistently that a Fourier dictionary yields the best results. We also demonstrate how the proposed method yields graphical interpretability in the form of phase portraits.
△ Less
Submitted 22 November, 2020;
originally announced November 2020.
-
Machine Learning a Molecular Hamiltonian for Predicting Electron Dynamics
Authors:
Harish S. Bhat,
Karnamohit Ranka,
Christine M. Isborn
Abstract:
We develop a computational method to learn a molecular Hamiltonian matrix from matrix-valued time series of the electron density. As we demonstrate for three small molecules, the resulting Hamiltonians can be used for electron density evolution, producing highly accurate results even when propagating 1000 time steps beyond the training data. As a more rigorous test, we use the learned Hamiltonians…
▽ More
We develop a computational method to learn a molecular Hamiltonian matrix from matrix-valued time series of the electron density. As we demonstrate for three small molecules, the resulting Hamiltonians can be used for electron density evolution, producing highly accurate results even when propagating 1000 time steps beyond the training data. As a more rigorous test, we use the learned Hamiltonians to simulate electron dynamics in the presence of an applied electric field, extrapolating to a problem that is beyond the field-free training data. We find that the resulting electron dynamics predicted by our learned Hamiltonian are in close quantitative agreement with the ground truth. Our method relies on combining a reduced-dimensional, linear statistical model of the Hamiltonian with a time-discretization of the quantum Liouville equation within time-dependent Hartree Fock theory. We train the model using a least-squares solver, avoiding numerous, CPU-intensive optimization steps. For both field-free and field-on problems, we quantify training and propagation errors, highlighting areas for future development.
△ Less
Submitted 31 August, 2020; v1 submitted 19 July, 2020;
originally announced July 2020.
-
Learning and Interpreting Potentials for Classical Hamiltonian Systems
Authors:
Harish S. Bhat
Abstract:
We consider the problem of learning an interpretable potential energy function from a Hamiltonian system's trajectories. We address this problem for classical, separable Hamiltonian systems. Our approach first constructs a neural network model of the potential and then applies an equation discovery technique to extract from the neural potential a closed-form algebraic expression. We demonstrate th…
▽ More
We consider the problem of learning an interpretable potential energy function from a Hamiltonian system's trajectories. We address this problem for classical, separable Hamiltonian systems. Our approach first constructs a neural network model of the potential and then applies an equation discovery technique to extract from the neural potential a closed-form algebraic expression. We demonstrate this approach for several systems, including oscillators, a central force problem, and a problem of two charged particles in a classical Coulomb potential. Through these test problems, we show close agreement between learned neural potentials, the interpreted potentials we obtain after training, and the ground truth. In particular, for the central force problem, we show that our approach learns the correct effective potential, a reduced-order model of the system.
△ Less
Submitted 26 July, 2019;
originally announced July 2019.
-
A Block Coordinate Descent Proximal Method for Simultaneous Filtering and Parameter Estimation
Authors:
Ramin Raziperchikolaei,
Harish S. Bhat
Abstract:
We propose and analyze a block coordinate descent proximal algorithm (BCD-prox) for simultaneous filtering and parameter estimation of ODE models. As we show on ODE systems with up to d=40 dimensions, as compared to state-of-the-art methods, BCD-prox exhibits increased robustness (to noise, parameter initialization, and hyperparameters), decreased training times, and improved accuracy of both filt…
▽ More
We propose and analyze a block coordinate descent proximal algorithm (BCD-prox) for simultaneous filtering and parameter estimation of ODE models. As we show on ODE systems with up to d=40 dimensions, as compared to state-of-the-art methods, BCD-prox exhibits increased robustness (to noise, parameter initialization, and hyperparameters), decreased training times, and improved accuracy of both filtered states and estimated parameters. We show how BCD-prox can be used with multistep numerical discretizations, and we establish convergence of BCD-prox under hypotheses that include real systems of interest.
△ Less
Submitted 24 May, 2019; v1 submitted 15 October, 2018;
originally announced October 2018.
-
Predicting Adolescent Suicide Attempts with Neural Networks
Authors:
Harish S. Bhat,
Sidra J. Goldman-Mellor
Abstract:
Though suicide is a major public health problem in the US, machine learning methods are not commonly used to predict an individual's risk of attempting/committing suicide. In the present work, starting with an anonymized collection of electronic health records for 522,056 unique, California-resident adolescents, we develop neural network models to predict suicide attempts. We frame the problem as…
▽ More
Though suicide is a major public health problem in the US, machine learning methods are not commonly used to predict an individual's risk of attempting/committing suicide. In the present work, starting with an anonymized collection of electronic health records for 522,056 unique, California-resident adolescents, we develop neural network models to predict suicide attempts. We frame the problem as a binary classification problem in which we use a patient's data from 2006-2009 to predict either the presence (1) or absence (0) of a suicide attempt in 2010. After addressing issues such as severely imbalanced classes and the variable length of a patient's history, we build neural networks with depths varying from two to eight hidden layers. For test set observations where we have at least five ED/hospital visits' worth of data on a patient, our depth-4 model achieves a sensitivity of 0.703, specificity of 0.980, and AUC of 0.958.
△ Less
Submitted 1 December, 2017; v1 submitted 27 November, 2017;
originally announced November 2017.
