Showing 1–11 of 11 results for author: Welborn, M

OrbNet Denali: A machine learning potential for biological and organic chemistry with semi-empirical cost and DFT accuracy

Authors: Anders S. Christensen, Sai Krishna Sirumalla, Zhuoran Qiao, Michael B. O'Connor, Daniel G. A. Smith, Feizhi Ding, Peter J. Bygrave, Animashree Anandkumar, Matthew Welborn, Frederick R. Manby, Thomas F. Miller III

Abstract: We present OrbNet Denali, a machine learning model for electronic structure that is designed as a drop-in replacement for ground-state density functional theory (DFT) energy calculations. The model is a message-passing neural network that uses symmetry-adapted atomic orbital features from a low-cost quantum calculation to predict the energy of a molecule. OrbNet Denali is trained on a vast dataset… ▽ More We present OrbNet Denali, a machine learning model for electronic structure that is designed as a drop-in replacement for ground-state density functional theory (DFT) energy calculations. The model is a message-passing neural network that uses symmetry-adapted atomic orbital features from a low-cost quantum calculation to predict the energy of a molecule. OrbNet Denali is trained on a vast dataset of 2.3 million DFT calculations on molecules and geometries. This dataset covers the most common elements in bio- and organic chemistry (H, Li, B, C, N, O, F, Na, Mg, Si, P, S, Cl, K, Ca, Br, I) as well as charged molecules. OrbNet Denali is demonstrated on several well-established benchmark datasets, and we find that it provides accuracy that is on par with modern DFT methods while offering a speedup of up to three orders of magnitude. For the GMTKN55 benchmark set, OrbNet Denali achieves WTMAD-1 and WTMAD-2 scores of 7.19 and 9.84, on par with modern DFT functionals. For several GMTKN55 subsets, which contain chemical problems that are not present in the training set, OrbNet Denali produces a mean absolute error comparable to those of DFT methods. For the Hutchison conformers benchmark set, OrbNet Denali has a median correlation coefficient of R^2=0.90 compared to the reference DLPNO-CCSD(T) calculation, and R^2=0.97 compared to the method used to generate the training data (wB97X-D3/def2-TZVP), exceeding the performance of any other method with a similar cost. Similarly, the model reaches chemical accuracy for non-covalent interactions in the S66x10 dataset. For torsional profiles, OrbNet Denali reproduces the torsion profiles of wB97X-D3/def2-TZVP with an average MAE of 0.12 kcal/mol for the potential energy surfaces of the diverse fragments in the TorsionNet500 dataset. △ Less

Submitted 2 July, 2021; v1 submitted 1 July, 2021; originally announced July 2021.

Informing Geometric Deep Learning with Electronic Interactions to Accelerate Quantum Chemistry

Authors: Zhuoran Qiao, Anders S. Christensen, Matthew Welborn, Frederick R. Manby, Anima Anandkumar, Thomas F. Miller III

Abstract: Predicting electronic energies, densities, and related chemical properties can facilitate the discovery of novel catalysts, medicines, and battery materials. By develo** a physics-inspired equivariant neural network, we introduce a method to learn molecular representations based on the electronic interactions among atomic orbitals. Our method, OrbNet-Equi, leverages efficient tight-binding simul… ▽ More Predicting electronic energies, densities, and related chemical properties can facilitate the discovery of novel catalysts, medicines, and battery materials. By develo** a physics-inspired equivariant neural network, we introduce a method to learn molecular representations based on the electronic interactions among atomic orbitals. Our method, OrbNet-Equi, leverages efficient tight-binding simulations and learned map**s to recover high fidelity quantum chemical properties. OrbNet-Equi models a wide spectrum of target properties with an accuracy consistently better than standard machine learning methods and a speed orders of magnitude greater than density functional theory. Despite only using training samples collected from readily available small-molecule libraries, OrbNet-Equi outperforms traditional methods on comprehensive downstream benchmarks that encompass diverse main-group chemical processes. Our method also describes interactions in challenging charge-transfer complexes and open-shell systems. We anticipate that the strategy presented here will help to expand opportunities for studies in chemistry and materials science, where the acquisition of experimental or reference training data is costly. △ Less

Submitted 1 April, 2022; v1 submitted 30 May, 2021; originally announced May 2021.

Journal ref: Proceedings of the National Academy of Sciences 119.31 (2022): e2205221119

Multi-task learning for electronic structure to predict and explore molecular potential energy surfaces

Authors: Zhuoran Qiao, Feizhi Ding, Matthew Welborn, Peter J. Bygrave, Daniel G. A. Smith, Animashree Anandkumar, Frederick R. Manby, Thomas F. Miller III

Abstract: We refine the OrbNet model to accurately predict energy, forces, and other response properties for molecules using a graph neural-network architecture based on features from low-cost approximated quantum operators in the symmetry-adapted atomic orbital basis. The model is end-to-end differentiable due to the derivation of analytic gradients for all electronic structure terms, and is shown to be tr… ▽ More We refine the OrbNet model to accurately predict energy, forces, and other response properties for molecules using a graph neural-network architecture based on features from low-cost approximated quantum operators in the symmetry-adapted atomic orbital basis. The model is end-to-end differentiable due to the derivation of analytic gradients for all electronic structure terms, and is shown to be transferable across chemical space due to the use of domain-specific features. The learning efficiency is improved by incorporating physically motivated constraints on the electronic structure through multi-task learning. The model outperforms existing methods on energy prediction tasks for the QM9 dataset and for molecular geometry optimizations on conformer datasets, at a computational cost that is thousand-fold or more reduced compared to conventional quantum-chemistry calculations (such as density functional theory) that offer similar accuracy. △ Less

Submitted 1 December, 2020; v1 submitted 5 November, 2020; originally announced November 2020.

Comments: Accepted for presentation at the Machine Learning for Molecules workshop at NeurIPS 2020

OrbNet: Deep Learning for Quantum Chemistry Using Symmetry-Adapted Atomic-Orbital Features

Authors: Zhuoran Qiao, Matthew Welborn, Animashree Anandkumar, Frederick R. Manby, Thomas F. Miller III

Abstract: We introduce a machine learning method in which energy solutions from the Schrodinger equation are predicted using symmetry adapted atomic orbitals features and a graph neural-network architecture. \textsc{OrbNet} is shown to outperform existing methods in terms of learning efficiency and transferability for the prediction of density functional theory results while employing low-cost features that… ▽ More We introduce a machine learning method in which energy solutions from the Schrodinger equation are predicted using symmetry adapted atomic orbitals features and a graph neural-network architecture. \textsc{OrbNet} is shown to outperform existing methods in terms of learning efficiency and transferability for the prediction of density functional theory results while employing low-cost features that are obtained from semi-empirical electronic structure calculations. For applications to datasets of drug-like molecules, including QM7b-T, QM9, GDB-13-T, DrugBank, and the conformer benchmark dataset of Folmsbee and Hutchison, \textsc{OrbNet} predicts energies within chemical accuracy of DFT at a computational cost that is thousand-fold or more reduced. △ Less

Submitted 18 January, 2022; v1 submitted 15 July, 2020; originally announced July 2020.

Journal ref: J. Chem. Phys. 153, 124111 (2020)

Regression-clustering for Improved Accuracy and Training Cost with Molecular-Orbital-Based Machine Learning

Authors: Lixue Cheng, Nikola B. Kovachki, Matthew Welborn, Thomas F. Miller III

Abstract: Machine learning (ML) in the representation of molecular-orbital-based (MOB) features has been shown to be an accurate and transferable approach to the prediction of post-Hartree-Fock correlation energies. Previous applications of MOB-ML employed Gaussian Process Regression (GPR), which provides good prediction accuracy with small training sets; however, the cost of GPR training scales cubically w… ▽ More Machine learning (ML) in the representation of molecular-orbital-based (MOB) features has been shown to be an accurate and transferable approach to the prediction of post-Hartree-Fock correlation energies. Previous applications of MOB-ML employed Gaussian Process Regression (GPR), which provides good prediction accuracy with small training sets; however, the cost of GPR training scales cubically with the amount of data and becomes a computational bottleneck for large training sets. In the current work, we address this problem by introducing a clustering/regression/classification implementation of MOB-ML. In a first step, regression clustering (RC) is used to partition the training data to best fit an ensemble of linear regression (LR) models; in a second step, each cluster is regressed independently, using either LR or GPR; and in a third step, a random forest classifier (RFC) is trained for the prediction of cluster assignments based on MOB feature values. Upon inspection, RC is found to recapitulate chemically intuitive grou**s of the frontier molecular orbitals, and the combined RC/LR/RFC and RC/GPR/RFC implementations of MOB-ML are found to provide good prediction accuracy with greatly reduced wall-clock training times. For a dataset of thermalized geometries of 7211 organic molecules of up to seven heavy atoms, both implementations reach chemical accuracy (1 kcal/mol error) with only 300 training molecules, while providing 35000-fold and 4500-fold reductions in the wall-clock training time, respectively, compared to MOB-ML without clustering. The resulting models are also demonstrated to retain transferability for the prediction of large-molecule energies with only small-molecule training data. Finally, it is shown that cap** the number of training datapoints per cluster leads to further improvements in prediction accuracy with negligible increases in wall-clock training time. △ Less

Submitted 23 October, 2019; v1 submitted 4 September, 2019; originally announced September 2019.

Comments: 31 pages, 10 figures, with an SI

A Universal Density Matrix Functional from Molecular Orbital-Based Machine Learning: Transferability across Organic Molecules

Authors: Lixue Cheng, Matthew Welborn, Anders S. Christensen, Thomas F. Miller III

Abstract: We address the degree to which machine learning can be used to accurately and transferably predict post-Hartree-Fock correlation energies. Refined strategies for feature design and selection are presented, and the molecular-orbital-based machine learning (MOB-ML) method is applied to several test systems. Strikingly, for the MP2, CCSD, and CCSD(T) levels of theory, it is shown that the thermally a… ▽ More We address the degree to which machine learning can be used to accurately and transferably predict post-Hartree-Fock correlation energies. Refined strategies for feature design and selection are presented, and the molecular-orbital-based machine learning (MOB-ML) method is applied to several test systems. Strikingly, for the MP2, CCSD, and CCSD(T) levels of theory, it is shown that the thermally accessible (350 K) potential energy surface for a single water molecule can be described to within 1 millihartree using a model that is trained from only a single reference calculation at a randomized geometry. To explore the breadth of chemical diversity that can be described, MOB-ML is also applied to a new dataset of thermalized (350 K) geometries of 7211 organic models with up to seven heavy atoms. In comparison with the previously reported $Δ$-ML method, MOB-ML is shown to reach chemical accuracy with three-fold fewer training geometries. Finally, a transferability test in which models trained for seven-heavy-atom systems are used to predict energies for thirteen-heavy-atom systems reveals that MOB-ML reaches chemical accuracy with 36-fold fewer training calculations than $Δ$-ML (140 versus 5000 training calculations). △ Less

Submitted 4 April, 2019; v1 submitted 10 January, 2019; originally announced January 2019.

Comments: 8 pages, 3 figures

Journal ref: J. Chem. Phys. 150, 131103 (2019)

Even-handed subsystem selection in projection-based embedding

Authors: Matthew Welborn, Frederick R. Manby, Thomas F. Miller III

Abstract: Projection-based embedding offers a simple framework for embedding correlated wavefunction methods in density functional theory. Partitioning between the correlated wavefunction and density functional subsystems is performed in the space of localized molecular orbitals. However, during a large geometry change--such as a chemical reaction--the nature of these localized molecular orbitals, as well a… ▽ More Projection-based embedding offers a simple framework for embedding correlated wavefunction methods in density functional theory. Partitioning between the correlated wavefunction and density functional subsystems is performed in the space of localized molecular orbitals. However, during a large geometry change--such as a chemical reaction--the nature of these localized molecular orbitals, as well as their partitioning into the two subsystems, can change dramatically. This can lead to unphysical cusps and even discontinuities in the potential energy surface. In this work, we present an even-handed framework for localized orbital partitioning that ensures consistent subsystems across a set of molecular geometries. We illustrate this problem and the even-handed solution with a simple example of an SN2 reaction. Applications to a nitrogen umbrella flip in a cobalt-based CO2 reduction catalyst and to the binding of CO to Cu clusters are presented. In both cases, we find that even-handed partitioning enables chemically accurate embedding with modestly-sized embedded regions for systems in which previous partitioning strategies are problematic. △ Less

Submitted 9 September, 2018; originally announced September 2018.

Comments: 11 pages, 5 figures

Journal ref: J. Chem. Phys., 2018, 149, 144101

Incremental Embedding: A Density Matrix Embedding Scheme for Molecules

Authors: Hong-Zhou Ye, Matthew Welborn, Nathan D. Ricke, Troy Van Voorhis

Abstract: The idea of using fragment embedding to circumvent the high computational scaling of accurate electronic structure methods while retaining high accuracy has been a long-standing goal for quantum chemists. Traditional fragment embedding methods mainly focus on systems composed of weakly correlated parts and are insufficient when division across chemical bonds is unavoidable. Recently, density matri… ▽ More The idea of using fragment embedding to circumvent the high computational scaling of accurate electronic structure methods while retaining high accuracy has been a long-standing goal for quantum chemists. Traditional fragment embedding methods mainly focus on systems composed of weakly correlated parts and are insufficient when division across chemical bonds is unavoidable. Recently, density matrix embedding theory (DMET) and other methods based on the Schmidt decomposition have emerged as a fresh approach to this problem. Despite their success on model systems, these methods can prove difficult for realistic systems because they rely on either a rigid, non-overlap** partition of the system or a specification of some special sites (i.e. `edge' and `center' sites), neither of which is well-defined in general for real molecules. In this work, we present a new Schmidt decomposition-based embedding scheme called Incremental Embedding that allows the combination of arbitrary overlap** fragments without the knowledge of edge sites. This method forms a convergent hierarchy in the sense that higher accuracy can be obtained by using fragments involving more sites. The computational scaling for the first few levels is lower than that of most correlated wave function methods. We present results for several small molecules in atom-centered Gaussian basis sets and demonstrate that Incremental Embedding converges quickly with fragment size and recovers most static correlation in small basis sets even when truncated at the second lowest level. △ Less

Submitted 23 July, 2018; originally announced July 2018.

Comments: 13 pages, 6 figures

Journal ref: J. Chem. Phys. 149, 194108 (2018)

Transferability in Machine Learning for Electronic Structure via the Molecular Orbital Basis

Authors: Matthew Welborn, Lixue Cheng, Thomas F. Miller III

Abstract: We present a machine learning (ML) method for predicting electronic structure correlation energies using Hartree-Fock input.The total correlation energy is expressed in terms of individual and pair contributions from occupied molecular orbitals, and Gaussian process regression is used to predict these contributions from a feature set that is based on molecular orbital properties, such as Fock, Cou… ▽ More We present a machine learning (ML) method for predicting electronic structure correlation energies using Hartree-Fock input.The total correlation energy is expressed in terms of individual and pair contributions from occupied molecular orbitals, and Gaussian process regression is used to predict these contributions from a feature set that is based on molecular orbital properties, such as Fock, Coulomb, and exchange matrix elements. With the aim of maximizing transferability across chemical systems and compactness of the feature set, we avoid the usual specification of ML features in terms of atom- or geometry-specific information, such atom/element-types, bond-types, or local molecular structure. ML predictions of MP2 and CCSD energies are presented for a range of systems, demonstrating that the method maintains accuracy while providing transferability both within and across chemical families; this includes predictions for molecules with atom-types and elements that are not included in the training set. The method holds promise both in its current form and as a proof-of-principle for the use of ML in the design of generalized density-matrix functionals. △ Less

Submitted 26 July, 2018; v1 submitted 31 May, 2018; originally announced June 2018.

Comments: 8 pages, 5 figures

Journal ref: J. Chem. Theory Comput., 2018, 14 (9), 4772

Accurate densities of states for disordered systems from free probability: Live Free or Diagonalize

Authors: Matthew Welborn, Jiahao Chen, Troy Van Voorhis

Abstract: We investigate how free probability allows us to approximate the density of states in tight binding models of disordered electronic systems. Extending our previous studies of the Anderson model in neighbor interactions [J. Chen et al., Phys. Rev. Lett. 109, 036403 (2012)], we find that free probability continues to provide accurate approximations for systems with constant interactions on two- and… ▽ More We investigate how free probability allows us to approximate the density of states in tight binding models of disordered electronic systems. Extending our previous studies of the Anderson model in neighbor interactions [J. Chen et al., Phys. Rev. Lett. 109, 036403 (2012)], we find that free probability continues to provide accurate approximations for systems with constant interactions on two- and three-dimensional lattices or with next-nearest-neighbor interactions, with the results being visually indistinguishable from the numerically exact solution. For systems with disordered interactions, we observe a small but visible degradation of the approximation. To explain this behavior of the free approximation, we develop and apply an asymptotic error analysis scheme to show that the approximation is accurate to the eighth moment in the density of states for systems with constant interactions, but is only accurate to sixth order for systems with disordered interactions. The error analysis also allows us to calculate asymptotic corrections to the density of states, allowing for systematically improvable approximations as well as insight into the sources of error without requiring a direct comparison to an exact solution. △ Less

Submitted 16 May, 2013; originally announced May 2013.

MSC Class: 70

Journal ref: Phys. Rev. B 88, 205113 (2013)

Error analysis of free probability approximations to the density of states of disordered systems

Authors: Jiahao Chen, Eric Hontz, Jeremy Moix, Matthew Welborn, Troy Van Voorhis, Alberto Suárez, Ramis Movassagh, Alan Edelman

Abstract: Theoretical studies of localization, anomalous diffusion and ergodicity breaking require solving the electronic structure of disordered systems. We use free probability to approximate the ensemble- averaged density of states without exact diagonalization. We present an error analysis that quantifies the accuracy using a generalized moment expansion, allowing us to distinguish between different app… ▽ More Theoretical studies of localization, anomalous diffusion and ergodicity breaking require solving the electronic structure of disordered systems. We use free probability to approximate the ensemble- averaged density of states without exact diagonalization. We present an error analysis that quantifies the accuracy using a generalized moment expansion, allowing us to distinguish between different approximations. We identify an approximation that is accurate to the eighth moment across all noise strengths, and contrast this with the perturbation theory and isotropic entanglement theory. △ Less

Submitted 27 February, 2012; originally announced February 2012.

Comments: 5 pages, 3 figures, submitted to Phys. Rev. Lett

MSC Class: 46L54; 65C60; 97K70

Journal ref: Phys. Rev. Lett. 109, 036403 (2012)