-
Tree-based variational inference for Poisson log-normal models
Authors:
Alexandre Chaussard,
Anna Bonnet,
Elisabeth Gassiat,
Sylvain Le Corff
Abstract:
When studying ecosystems, hierarchical trees are often used to organize entities based on proximity criteria, such as the taxonomy in microbiology, social classes in geography, or product types in retail businesses, offering valuable insights into entity relationships. Despite their significance, current count-data models do not leverage this structured information. In particular, the widely used…
▽ More
When studying ecosystems, hierarchical trees are often used to organize entities based on proximity criteria, such as the taxonomy in microbiology, social classes in geography, or product types in retail businesses, offering valuable insights into entity relationships. Despite their significance, current count-data models do not leverage this structured information. In particular, the widely used Poisson log-normal (PLN) model, known for its ability to model interactions between entities from count data, lacks the possibility to incorporate such hierarchical tree structures, limiting its applicability in domains characterized by such complexities. To address this matter, we introduce the PLN-Tree model as an extension of the PLN model, specifically designed for modeling hierarchical count data. By integrating structured variational inference techniques, we propose an adapted training procedure and establish identifiability results, enhancisng both theoretical foundations and practical interpretability. Additionally, we extend our framework to classification tasks as a preprocessing pipeline, showcasing its versatility. Experimental evaluations on synthetic datasets as well as real-world microbiome data demonstrate the superior performance of the PLN-Tree model in capturing hierarchical dependencies and providing valuable insights into complex data structures, showing the practical interest of knowledge graphs like the taxonomy in ecosystems modeling.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
Spectral analysis for noisy Hawkes processes inference
Authors:
Anna Bonnet,
Felix Cheysson,
Miguel Martinez Herrera,
Maxime Sangnier
Abstract:
Classic estimation methods for Hawkes processes rely on the assumption that observed event times are indeed a realisation of a Hawkes process, without considering any potential perturbation of the model. However, in practice, observations are often altered by some noise, the form of which depends on the context.It is then required to model the alteration mechanism in order to infer accurately such…
▽ More
Classic estimation methods for Hawkes processes rely on the assumption that observed event times are indeed a realisation of a Hawkes process, without considering any potential perturbation of the model. However, in practice, observations are often altered by some noise, the form of which depends on the context.It is then required to model the alteration mechanism in order to infer accurately such a noisy Hawkes process. While several models exist, we consider, in this work, the observations to be the indistinguishable union of event times coming from a Hawkes process and from an independent Poisson process. Since standard inference methods (such as maximum likelihood or Expectation-Maximisation) are either unworkable or numerically prohibitive in this context, we propose an estimation procedure based on the spectral analysis of second order properties of the noisy Hawkes process. Novel results include sufficient conditions for identifiability of the ensuing statistical model with exponential interaction functions for both univariate and bivariate processes. Although we mainly focus on the exponential scenario, other types of kernels are investigated and discussed. A new estimator based on maximising the spectral log-likelihood is then described, and its behaviour is numerically illustrated on synthetic data. Besides being free from knowing the source of each observed time (Hawkes or Poisson process), the proposed estimator is shown to perform accurately in estimating both processes.
△ Less
Submitted 21 May, 2024;
originally announced May 2024.
-
MEDITRON-70B: Scaling Medical Pretraining for Large Language Models
Authors:
Zeming Chen,
Alejandro Hernández Cano,
Angelika Romanou,
Antoine Bonnet,
Kyle Matoba,
Francesco Salvi,
Matteo Pagliardini,
Simin Fan,
Andreas Köpf,
Amirkeivan Mohtashami,
Alexandre Sallinen,
Alireza Sakhaeirad,
Vinitra Swamy,
Igor Krawczuk,
Deniz Bayazit,
Axel Marmet,
Syrielle Montariol,
Mary-Anne Hartley,
Martin Jaggi,
Antoine Bosselut
Abstract:
Large language models (LLMs) can potentially democratize access to medical knowledge. While many efforts have been made to harness and improve LLMs' medical knowledge and reasoning capacities, the resulting models are either closed-source (e.g., PaLM, GPT-4) or limited in scale (<= 13B parameters), which restricts their abilities. In this work, we improve access to large-scale medical LLMs by rele…
▽ More
Large language models (LLMs) can potentially democratize access to medical knowledge. While many efforts have been made to harness and improve LLMs' medical knowledge and reasoning capacities, the resulting models are either closed-source (e.g., PaLM, GPT-4) or limited in scale (<= 13B parameters), which restricts their abilities. In this work, we improve access to large-scale medical LLMs by releasing MEDITRON: a suite of open-source LLMs with 7B and 70B parameters adapted to the medical domain. MEDITRON builds on Llama-2 (through our adaptation of Nvidia's Megatron-LM distributed trainer), and extends pretraining on a comprehensively curated medical corpus, including selected PubMed articles, abstracts, and internationally-recognized medical guidelines. Evaluations using four major medical benchmarks show significant performance gains over several state-of-the-art baselines before and after task-specific finetuning. Overall, MEDITRON achieves a 6% absolute performance gain over the best public baseline in its parameter class and 3% over the strongest baseline we finetuned from Llama-2. Compared to closed-source LLMs, MEDITRON-70B outperforms GPT-3.5 and Med-PaLM and is within 5% of GPT-4 and 10% of Med-PaLM-2. We release our code for curating the medical pretraining corpus and the MEDITRON model weights to drive open-source development of more capable medical LLMs.
△ Less
Submitted 27 November, 2023;
originally announced November 2023.
-
Computational Reproducibility in Metabolite Quantification Applied to Short Echo Time in Vivo MR Spectroscopy
Authors:
Gaël Vila,
Axel Bonnet,
Fabien Chauveau,
Sophie Gaillard,
Françoise Durand-Dubief,
Sorina Camarasu-Pop,
Helene Ratiney
Abstract:
In vivo metabolite quantification by short echo time MR spectroscopy is a challenge for which various methods have been proposed. In this study, the reproducibility of quantification outcomes is questioned at three distinct levels: (i) between-software (LCModel and cQUEST), (ii) withinsoftware (with different parameter sets), and (iii) across software executions (when the fitting algorithm uses ra…
▽ More
In vivo metabolite quantification by short echo time MR spectroscopy is a challenge for which various methods have been proposed. In this study, the reproducibility of quantification outcomes is questioned at three distinct levels: (i) between-software (LCModel and cQUEST), (ii) withinsoftware (with different parameter sets), and (iii) across software executions (when the fitting algorithm uses random seeds, like cQUEST). After running multiple quantification tasks on a dedicated platform (VIP), metrics from Bland-Altman analysis were used to assess the variability of outcomes in signals acquired on a lysolecithin rat model, from a study on demyelination. Results show substantial variations at the three levels, allowing for more potent analyses than from a single parameter set / single software point of view.
△ Less
Submitted 9 March, 2023;
originally announced March 2023.
-
Inference of multivariate exponential Hawkes processes with inhibition and application to neuronal activity
Authors:
Anna Bonnet,
Miguel Martinez Herrera,
Maxime Sangnier
Abstract:
The multivariate Hawkes process is a past-dependent point process used to model the relationship of event occurrences between different phenomena.Although the Hawkes process was originally introduced to describe excitation effects, which means that one event increases the chances of another occurring, there has been a growing interest in modelling the opposite effect, known as inhibition.In this p…
▽ More
The multivariate Hawkes process is a past-dependent point process used to model the relationship of event occurrences between different phenomena.Although the Hawkes process was originally introduced to describe excitation effects, which means that one event increases the chances of another occurring, there has been a growing interest in modelling the opposite effect, known as inhibition.In this paper, we focus on how to infer the parameters of a multidimensional exponential Hawkes process with both excitation and inhibition effects. Our first result is to prove the identifiability of this model under a few sufficient assumptions. Then we propose a maximum likelihood approach to estimate the interaction functions, which is, to the best of our knowledge, the first exact inference procedure in the frequentist framework.Our method includes a variable selection step in order to recover the support of interactions and therefore to infer the connectivity graph.A benefit of our method is to provide an explicit computation of the log-likelihood, which enables in addition to perform a goodness-of-fit test for assessing the quality of estimations.We compare our method to standard approaches, which were developed in the linear framework and are not specifically designed for handling inhibiting effects.We show that the proposed estimator performs better on synthetic data than alternative approaches. We also illustrate the application of our procedure to a neuronal activity dataset, which highlights the presence of both exciting and inhibiting effects between neurons.
△ Less
Submitted 29 June, 2023; v1 submitted 9 May, 2022;
originally announced May 2022.
-
Neuronal Network Inference and Membrane Potential Model using Multivariate Hawkes Processes
Authors:
Anna Bonnet,
Charlotte Dion,
François Gindraud,
Sarah Lemler
Abstract:
In this work, we propose to catch the complexity of the membrane potential's dynamic of a motoneuron between its spikes, taking into account the spikes from other neurons around. Our approach relies on two types of data: extracellular recordings of multiple spikes trains and intracellular recordings of the membrane potential of a central neuron. Our main contribution is to provide a unified framew…
▽ More
In this work, we propose to catch the complexity of the membrane potential's dynamic of a motoneuron between its spikes, taking into account the spikes from other neurons around. Our approach relies on two types of data: extracellular recordings of multiple spikes trains and intracellular recordings of the membrane potential of a central neuron. Our main contribution is to provide a unified framework and a complete pipeline to analyze neuronal activity from data extraction to statistical inference. The first step of the procedure is to select a subnetwork of neurons impacting the central neuron: we use a multivariate Hawkes process to model the spike trains of all neurons and compare two sparse inference procedures to identify the connectivity graph. Then we infer a jump-diffusion dynamic in which jumps are driven from a Hawkes process, the occurrences of which correspond to the spike trains of the aforementioned subset of neurons that interact with the central neuron. We validate the Hawkes model with a goodness-of-fit test and we show that taking into account the information from the connectivity graph improves the inference of the jump-diffusion process. The entire code has been developed and is freely available on GitHub.
△ Less
Submitted 2 August, 2021;
originally announced August 2021.
-
Maximum Likelihood Estimation for Hawkes Processes with self-excitation or inhibition
Authors:
Anna Bonnet,
Miguel Martinez Herrera,
Maxime Sangnier
Abstract:
In this paper, we present a maximum likelihood method for estimating the parameters of a univariate Hawkes process with self-excitation or inhibition. Our work generalizes techniques and results that were restricted to the self-exciting scenario. The proposed estimator is implemented for the classical exponential kernel and we show that, in the inhibition context, our procedure provides more accur…
▽ More
In this paper, we present a maximum likelihood method for estimating the parameters of a univariate Hawkes process with self-excitation or inhibition. Our work generalizes techniques and results that were restricted to the self-exciting scenario. The proposed estimator is implemented for the classical exponential kernel and we show that, in the inhibition context, our procedure provides more accurate estimations than current alternative approaches.
△ Less
Submitted 20 August, 2021; v1 submitted 9 March, 2021;
originally announced March 2021.
-
Uniform Deconvolution for Poisson Point Processes
Authors:
Anna Bonnet,
Claire Lacour,
Franck Picard,
Vincent Rivoirard
Abstract:
We focus on the estimation of the intensity of a Poisson process in the presence of a uniform noise. We propose a kernel-based procedure fully calibrated in theory and practice. We show that our adaptive estimator is optimal from the oracle and minimax points of view, and provide new lower bounds when the intensity belongs to a Sobolev ball. By develo** the Goldenshluger-Lepski methodology in th…
▽ More
We focus on the estimation of the intensity of a Poisson process in the presence of a uniform noise. We propose a kernel-based procedure fully calibrated in theory and practice. We show that our adaptive estimator is optimal from the oracle and minimax points of view, and provide new lower bounds when the intensity belongs to a Sobolev ball. By develo** the Goldenshluger-Lepski methodology in the case of deconvolution for Poisson processes, we propose an optimal data-driven selection of the kernel bandwidth. Our method is illustrated on the spatial distribution of replication origins and sequence motifs along the human genome.
△ Less
Submitted 28 June, 2022; v1 submitted 9 October, 2020;
originally announced October 2020.
-
Boutiques: a flexible framework for automated application integration in computing platforms
Authors:
Tristan Glatard,
Gregory Kiar,
Tristan Aumentado-Armstrong,
Natacha Beck,
Pierre Bellec,
Rémi Bernard,
Axel Bonnet,
Sorina Camarasu-Pop,
Frédéric Cervenansky,
Samir Das,
Rafael Ferreira da Silva,
Guillaume Flandin,
Pascal Girard,
Krzysztof J. Gorgolewski,
Charles R. G. Guttmann,
Valérie Hayot-Sasson,
Pierre-Olivier Quirion,
Pierre Rioux,
Marc-Eienne Rousseau,
Alan C. Evans
Abstract:
We present Boutiques, a system to automatically publish, integrate and execute applications across computational platforms. Boutiques applications are installed through software containers described in a rich and flexible JSON language. A set of core tools facilitate the construction, validation, import, execution, and publishing of applications. Boutiques is currently supported by several distinc…
▽ More
We present Boutiques, a system to automatically publish, integrate and execute applications across computational platforms. Boutiques applications are installed through software containers described in a rich and flexible JSON language. A set of core tools facilitate the construction, validation, import, execution, and publishing of applications. Boutiques is currently supported by several distinct virtual research platforms, and it has been used to describe dozens of applications in the neuroinformatics domain. We expect Boutiques to improve the quality of application integration in computational platforms, to reduce redundancy of effort, to contribute to computational reproducibility, and to foster Open Science.
△ Less
Submitted 7 November, 2017;
originally announced November 2017.
-
Heritability estimation of diseases in case-control studies
Authors:
Anna Bonnet
Abstract:
In the field of genetics, the concept of heritability refers to the proportion of variations of a biological trait or disease that can be explained by genetic factors. Quantifying the heritability of a disease is a fundamental challenge in human genetics, especially when the causes are plural and not clearly identified. Although the literature regarding heritability estimation for binary traits is…
▽ More
In the field of genetics, the concept of heritability refers to the proportion of variations of a biological trait or disease that can be explained by genetic factors. Quantifying the heritability of a disease is a fundamental challenge in human genetics, especially when the causes are plural and not clearly identified. Although the literature regarding heritability estimation for binary traits is less rich than for quantitative traits, several methods have been proposed to estimate the heritability of complex diseases. However, to the best of our knowledge, the existing methods are not supported by theoretical grounds. Moreover, most of the methodologies do not take into account a major specificity of the data coming from medical studies, which is the oversampling of the number of patients compared to controls. We propose in this paper to investigate the theoretical properties of the method developed by Golan et al. (2014), which is very efficient in practice, despite the oversampling of patients. Our main result is the proof of the consistency of this estimator. We also provide a numerical study to compare two approximations leading to two heritability estimators.
△ Less
Submitted 9 November, 2016;
originally announced November 2016.
-
Improving heritability estimation by a variable selection approach in sparse high dimensional linear mixed models
Authors:
Anna Bonnet,
Céline Lévy-Leduc,
Elisabeth Gassiat,
Roberto Toro,
Thomas Bourgeron
Abstract:
Motivated by applications in neuroanatomy, we propose a novel methodology for estimating the heritability which corresponds to the proportion of phenotypic variance which can be explained by genetic factors. Estimating this quantity for neuroanatomical features is a fundamental challenge in psychiatric disease research. Since the phenotypic variations may only be due to a small fraction of the ava…
▽ More
Motivated by applications in neuroanatomy, we propose a novel methodology for estimating the heritability which corresponds to the proportion of phenotypic variance which can be explained by genetic factors. Estimating this quantity for neuroanatomical features is a fundamental challenge in psychiatric disease research. Since the phenotypic variations may only be due to a small fraction of the available genetic information, we propose an estimator of the heritability that can be used in high dimensional sparse linear mixed models. Our method consists of three steps. Firstly, a variable selection stage is performed in order to recover the support of the genetic effects -- also called causal variants -- that is to find the genetic effects which really explain the phenotypic variations. Secondly, we propose a maximum likelihood strategy for estimating the heritability which only takes into account the causal genetic effects found in the first step. Thirdly, we compute the standard error and the 95% confidence interval associated to our heritability estimator thanks to a nonparametric bootsrap approach. Our main contribution consists in providing an estimation of the heritability with standard errors substantially smaller than methods without variable selection when the genetic effects are very sparse. Since the real genetic architecture is in general unknown in practice, we also propose an empirical criterion which allows the user to decide whether it is relevant to apply a variable selection based approach or not. We illustrate the performance of our methodology on synthetic and real neuroanatomic data coming from the Imagen project. We also show that our approach has a very low computational burden and is very efficient from a statistical point of view.
△ Less
Submitted 8 June, 2016; v1 submitted 22 July, 2015;
originally announced July 2015.
-
Heritability estimation in high dimensional linear mixed models
Authors:
Anna Bonnet,
Elisabeth Gassiat,
Céline Lévy-Leduc
Abstract:
Motivated by applications in genetic fields, we propose to estimate the heritability in high dimensional sparse linear mixed models. The heritability determines how the variance is shared between the different random components of a linear mixed model. The main novelty of our approach is to consider that the random effects can be sparse, that is may contain null components, but we do not know neit…
▽ More
Motivated by applications in genetic fields, we propose to estimate the heritability in high dimensional sparse linear mixed models. The heritability determines how the variance is shared between the different random components of a linear mixed model. The main novelty of our approach is to consider that the random effects can be sparse, that is may contain null components, but we do not know neither their proportion nor their positions. The estimator that we consider is strongly inspired by the one proposed by Pirinen et al. (2013), and is based on a maximum likelihood approach. We also study the theoretical properties of our estimator, namely we establish that our estimator of the heritability is $\sqrt{n}$-consistent when both the number of observations $n$ and the number of random effects $N$ tend to infinity under mild assumptions. We also prove that our estimator of the heritability satisfies a central limit theorem which gives as a byproduct a confidence interval for the heritability. Some Monte-Carlo experiments are also conducted in order to show the finite sample performances of our estimator.
△ Less
Submitted 6 May, 2015; v1 submitted 13 April, 2014;
originally announced April 2014.
-
Dynamical quark mass generation in a strong external magnetic field
Authors:
Niklas Mueller,
Jacqueline A. Bonnet,
Christian S. Fischer
Abstract:
We investigate the effect of a strong magnetic field on dynamical chiral symmetry breaking in quenched and unquenched QCD. To this end we apply the Ritus formalism to the coupled set of (truncated) Dyson-Schwinger equations for the quark and gluon propagator under the presence of an external constant Abelian magnetic field. We work with an approximation that is trustworthy for large fields eH > Λ_…
▽ More
We investigate the effect of a strong magnetic field on dynamical chiral symmetry breaking in quenched and unquenched QCD. To this end we apply the Ritus formalism to the coupled set of (truncated) Dyson-Schwinger equations for the quark and gluon propagator under the presence of an external constant Abelian magnetic field. We work with an approximation that is trustworthy for large fields eH > Λ_{QCD}^2 but is not restricted to the lowest Landau level. We confirm the linear rise of the quark condensate with large external field previously found in other studies and observe the transition to the asymptotic power law at extremely large fields. We furthermore quantify the validity of the lowest Landau level approximation and find substantial quantitative differences to the full calculation even at very large fields. We discuss unquenching effects in the strong field propagators, condensate and the magnetic polarization of the vacuum. We find a significant weakening of magnetic catalysis caused by the back reaction of quarks on the Yang-Mills sector. Our results support explanations of the inverse magnetic catalysis found in recent lattice studies due to unquenching effects.
△ Less
Submitted 7 February, 2014; v1 submitted 8 January, 2014;
originally announced January 2014.
-
Critical scaling of finite temperature QED_3 in anisotropic space-time
Authors:
Jacqueline A. Bonnet,
Christian S. Fischer
Abstract:
We investigate the scaling behavior of the critical temperature of anisotropic QED in 2+1 dimensions with respect to a variation of the number of fermions N_f. To this end we determine the order parameter of the chiral transition of the theory from a set of (truncated) Dyson-Schwinger equations for the fermion propagator formulated in a finite volume. We verify the validity of previously determine…
▽ More
We investigate the scaling behavior of the critical temperature of anisotropic QED in 2+1 dimensions with respect to a variation of the number of fermions N_f. To this end we determine the order parameter of the chiral transition of the theory from a set of (truncated) Dyson-Schwinger equations for the fermion propagator formulated in a finite volume. We verify the validity of previously determined universal power laws for the scaling behavior of the critical temperature with N_f. We furthermore study the variation of the corresponding critical exponent with the degree of anisotropy and find considerable variations.
△ Less
Submitted 9 November, 2012; v1 submitted 30 January, 2012;
originally announced January 2012.
-
Effects of Anisotropy in (2+1)-dimensional QED
Authors:
J. A. Bonnet,
C. S. Fischer,
R. Williams
Abstract:
We summarize our results for the impact of anisotropic fermionic velocities in (2+1)-dimensional QED on the critical number of fermion flavors, N^c_f, and dynamical mass generation. We apply different approximation schemes for the gauge boson vacuum polarization and the fermion-boson vertex to analyze the according Dyson-Schwinger equations in a finite volume. Our results point towards large varia…
▽ More
We summarize our results for the impact of anisotropic fermionic velocities in (2+1)-dimensional QED on the critical number of fermion flavors, N^c_f, and dynamical mass generation. We apply different approximation schemes for the gauge boson vacuum polarization and the fermion-boson vertex to analyze the according Dyson-Schwinger equations in a finite volume. Our results point towards large variations of N^c_f away from the isotropic point in agreement with other approaches.
△ Less
Submitted 1 November, 2011;
originally announced November 2011.
-
Effects of Anisotropy in QED3 from Dyson-Schwinger equations in a box
Authors:
Jacqueline A. Bonnet,
Christian S. Fischer,
Richard Williams
Abstract:
We investigate the effect of anisotropies in the fermion velocities of 2+1 dimensional QED on the critical number N_f^c of fermions for dynamical mass generation. Our framework are the Dyson-Schwinger equations for the gauge boson and fermion propagators formulated in a finite volume. In contrast to previous Dyson-Schwinger studies we do not rely on an expansion in small anisotropies but keep the…
▽ More
We investigate the effect of anisotropies in the fermion velocities of 2+1 dimensional QED on the critical number N_f^c of fermions for dynamical mass generation. Our framework are the Dyson-Schwinger equations for the gauge boson and fermion propagators formulated in a finite volume. In contrast to previous Dyson-Schwinger studies we do not rely on an expansion in small anisotropies but keep the full velocity dependence of fermion equations intact. As result we find sizable variations of N_f^c away from the isotropic point in agreement with other approaches. We discuss the relevance of our findings for models of high-T_c superconductors.
△ Less
Submitted 29 June, 2011; v1 submitted 8 March, 2011;
originally announced March 2011.
