Showing 1–3 of 3 results for author: Boesen, T

Neural DAEs: Constrained neural networks

Authors: Tue Boesen, Eldad Haber, Uri Michael Ascher

Abstract: This article investigates the effect of explicitly adding auxiliary algebraic trajectory information to neural networks for dynamical systems. We draw inspiration from the field of differential-algebraic equations and differential equations on manifolds and implement related methods in residual neural networks, despite some fundamental scenario differences. Constraint or auxiliary information effe… ▽ More This article investigates the effect of explicitly adding auxiliary algebraic trajectory information to neural networks for dynamical systems. We draw inspiration from the field of differential-algebraic equations and differential equations on manifolds and implement related methods in residual neural networks, despite some fundamental scenario differences. Constraint or auxiliary information effects are incorporated through stabilization as well as projection methods, and we show when to use which method based on experiments involving simulations of multi-body pendulums and molecular dynamics scenarios. Several of our methods are easy to implement in existing code and have limited impact on training performance while giving significant boosts in terms of inference. △ Less

Submitted 12 March, 2024; v1 submitted 25 November, 2022; originally announced November 2022.

Comments: Extended the paper to PDEs, added a third experiment denoising a vector field and updated the introduction to make the distinction between this work and physics informed neural networks more clear

MSC Class: 70H99; 34A09

A-Optimal Active Learning

Authors: Tue Boesen, Eldad Haber

Abstract: In this work we discuss the problem of active learning. We present an approach that is based on A-optimal experimental design of ill-posed problems and show how one can optimally label a data set by partially probing it, and use it to train a deep network. We present two approaches that make different assumptions on the data set. The first is based on a Bayesian interpretation of the semi-supervis… ▽ More In this work we discuss the problem of active learning. We present an approach that is based on A-optimal experimental design of ill-posed problems and show how one can optimally label a data set by partially probing it, and use it to train a deep network. We present two approaches that make different assumptions on the data set. The first is based on a Bayesian interpretation of the semi-supervised learning problem with the graph Laplacian that is used for the prior distribution and the second is based on a frequentist approach, that updates the estimation of the bias term based on the recovery of the labels. We demonstrate that this approach can be highly efficient for estimating labels and training a deep network. △ Less

Submitted 25 November, 2022; v1 submitted 18 October, 2021; originally announced October 2021.

Comments: 14 pages, submitted to Physica Scripta

MSC Class: 68T05 (primary); 68T07; 62K05

Mimetic Neural Networks: A unified framework for Protein Design and Folding

Authors: Moshe Eliasof, Tue Boesen, Eldad Haber, Chen Keasar, Eran Treister

Abstract: Recent advancements in machine learning techniques for protein folding motivate better results in its inverse problem -- protein design. In this work we introduce a new graph mimetic neural network, MimNet, and show that it is possible to build a reversible architecture that solves the structure and design problems in tandem, allowing to improve protein design when the structure is better estimate… ▽ More Recent advancements in machine learning techniques for protein folding motivate better results in its inverse problem -- protein design. In this work we introduce a new graph mimetic neural network, MimNet, and show that it is possible to build a reversible architecture that solves the structure and design problems in tandem, allowing to improve protein design when the structure is better estimated. We use the ProteinNet data set and show that the state of the art results in protein design can be improved, given recent architectures for protein folding. △ Less

Submitted 7 February, 2021; originally announced February 2021.