-
Inverse-Dirichlet Weighting Enables Reliable Training of Physics Informed Neural Networks
Authors:
Suryanarayana Maddu,
Dominik Sturm,
Christian L. Müller,
Ivo F. Sbalzarini
Abstract:
We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as Physics Informed Neural Networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of dat…
▽ More
We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as Physics Informed Neural Networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of data-fidelity and equation-fidelity objectives. Conflicts between objectives can arise from scale imbalances, heteroscedasticity in the data, stiffness of the physical equation, or from catastrophic interference during sequential training. We explain the training pathology arising from this and propose a simple yet effective inverse-Dirichlet weighting strategy to alleviate the issue. We compare with Sobolev training of neural networks, providing the baseline of analytically $\boldsymbolε$-optimal training. We demonstrate the effectiveness of inverse-Dirichlet weighting in various applications, including a multi-scale model of active turbulence, where we show orders of magnitude improvement in accuracy and convergence over conventional PINN training. For inverse modeling using sequential training, we find that inverse-Dirichlet weighting protects a PINN against catastrophic forgetting.
△ Less
Submitted 2 July, 2021;
originally announced July 2021.
-
Estimating state occupation and transition probabilities in non-Markov multi-state models subject to both random left-truncation and right-censoring
Authors:
Alexandra Niessl,
Arthur Allignol,
Carina Mueller,
Jan Beyersmann
Abstract:
The Aalen-Johansen estimator generalizes the Kaplan-Meier estimator for independently left-truncated and right-censored survival data to estimating the transition probability matrix of a time-inhomogeneous Markov model with finite state space. Such multi-state models have a wide range of applications for modelling complex courses of a disease over the course of time, but the Markov assumption may…
▽ More
The Aalen-Johansen estimator generalizes the Kaplan-Meier estimator for independently left-truncated and right-censored survival data to estimating the transition probability matrix of a time-inhomogeneous Markov model with finite state space. Such multi-state models have a wide range of applications for modelling complex courses of a disease over the course of time, but the Markov assumption may often be in doubt. If censoring is entirely unrelated to the multi-state data, it has been noted that the Aalen-Johansen estimator, standardized by the initial empirical distribution of the multi-state model, still consistently estimates the state occupation probabilities. Recently, this result has been extended to transition probabilities using landmarking, which is, inter alia, useful for dynamic prediction. We complement these results in three ways. Firstly, delayed study entry is a common phenomenon in observational studies, and we extend the earlier results to multi-state data also subject to left-truncation. Secondly, we present a rigorous proof of consistency of the Aalen-Johansen estimator for state occupation probabilities, on which also correctness of the landmarking approach hinges, correcting, simplifying and extending the earlier result. Thirdly, our rigorous proof motivates wild bootstrap resampling. Our developments for left-truncation are motivated by a prospective observational study on the occurrence and the impact of a multi-resistant infectious organism in patients undergoing surgery. Both the real data example and simulation studies are presented. Studying wild bootstrap is motivated by the fact that, unlike drawing with replacement from the data, it is desirable to have a technique that works both with non-Markov models subject to random left-truncation and right-censoring and with Markov models where left-truncation and right-censoring need not be entirely random.
△ Less
Submitted 14 April, 2020;
originally announced April 2020.
-
Robust Regression with Compositional Covariates
Authors:
Aditya Mishra,
Christian L. Muller
Abstract:
Many biological high-throughput data sets, such as targeted amplicon-based and metagenomic sequencing data, are compositional in nature. A common exploratory data analysis task is to infer statistical associations between the high-dimensional microbial compositions and habitat- or host-related covariates. We propose a general robust statistical regression framework, RobRegCC (Robust Regression wit…
▽ More
Many biological high-throughput data sets, such as targeted amplicon-based and metagenomic sequencing data, are compositional in nature. A common exploratory data analysis task is to infer statistical associations between the high-dimensional microbial compositions and habitat- or host-related covariates. We propose a general robust statistical regression framework, RobRegCC (Robust Regression with Compositional Covariates), which extends the linear log-contrast model by a mean shift formulation for capturing outliers. RobRegCC includes sparsity-promoting convex and non-convex penalties for parsimonious model estimation, a data-driven robust initialization procedure, and a novel robust cross-validation model selection scheme. We show RobRegCC's ability to perform simultaneous sparse log-contrast regression and outlier detection over a wide range of simulation settings and provide theoretical non-asymptotic guarantees for the underlying estimators. To demonstrate the seamless applicability of the workflow on real data, we consider a gut microbiome data set from HIV patients and infer robust associations between a sparse set of microbial species and host immune response from soluble CD14 measurements. All experiments are fully reproducible and available on GitHub at https://github.com/amishra-stats/robregcc.
△ Less
Submitted 26 July, 2020; v1 submitted 11 September, 2019;
originally announced September 2019.
-
Stacked search for time shifted high energy neutrinos from gamma ray bursts with the ANTARES neutrino telescope
Authors:
ANTARES Collaboration,
S. Adrian-Martínez,
A. Albert,
M. André,
M. Anghinolfi,
G. Anton,
M. Ardid,
J. -J. Aubert,
B. Baret,
J. Barrios-Marti,
S. Basa,
V. Bertin,
S. Biagi,
R. Bormuth,
M. C. Bouwhuis,
R. Bruijn,
J. Brunner,
J. Busto,
A. Capone,
L. Caramete,
J. Carr,
T. Chiarusi,
M. Circella,
R. Coniglione,
H. Costantini
, et al. (97 additional authors not shown)
Abstract:
A search for high-energy neutrino emission correlated with gamma-ray bursts outside the electromagnetic prompt-emission time window is presented. Using a stacking approach of the time delays between reported gamma-ray burst alerts and spatially coincident muon-neutrino signatures, data from the Antares neutrino telescope recorded between 2007 and 2012 are analysed. One year of public data from the…
▽ More
A search for high-energy neutrino emission correlated with gamma-ray bursts outside the electromagnetic prompt-emission time window is presented. Using a stacking approach of the time delays between reported gamma-ray burst alerts and spatially coincident muon-neutrino signatures, data from the Antares neutrino telescope recorded between 2007 and 2012 are analysed. One year of public data from the IceCube detector between 2008 and 2009 have been also investigated. The respective timing profiles are scanned for statistically significant accumulations within 40 days of the Gamma Ray Burst, as expected from Lorentz Invariance Violation effects and some astrophysical models. No significant excess over the expected accidental coincidence rate could be found in either of the two data sets. The average strength of the neutrino signal is found to be fainter than one detectable neutrino signal per hundred gamma-ray bursts in the Antares data at 90% confidence level.
△ Less
Submitted 20 October, 2016; v1 submitted 31 August, 2016;
originally announced August 2016.
-
Spectroscopy of Surface-Induced Noise Using Shallow Spins in Diamond
Authors:
Y. Romach,
C. Muller,
T. Unden,
L. J. Rogers,
T. Isoda,
K. M. Itoh,
M. Markham,
A. Stacey,
J. Meijer,
S. Pezzagna,
B. Naydenov,
L. P. McGuinness,
N. Bar-Gill,
F. Jelezko
Abstract:
We report on the noise spectrum experienced by few nanometer deep nitrogen-vacancy centers in diamond as a function of depth, surface coating, magnetic field and temperature. Analysis reveals a double-Lorentzian noise spectra consistent with a surface electronic spin bath, with slower dynamics due to spin-spin interactions and faster dynamics related to phononic coupling. These results shed new li…
▽ More
We report on the noise spectrum experienced by few nanometer deep nitrogen-vacancy centers in diamond as a function of depth, surface coating, magnetic field and temperature. Analysis reveals a double-Lorentzian noise spectra consistent with a surface electronic spin bath, with slower dynamics due to spin-spin interactions and faster dynamics related to phononic coupling. These results shed new light on the mechanisms responsible for surface noise affecting shallow spins at semiconductor interfaces, and suggests possible directions for further studies. We demonstrate dynamical decoupling from the surface noise, paving the way to applications ranging from nanoscale NMR to quantum networks.
△ Less
Submitted 25 February, 2015; v1 submitted 15 April, 2014;
originally announced April 2014.
