-
The Stochastic Occupation Kernel Method for System Identification
Authors:
Michael Wells,
Kamel Lahouel,
Bruno Jedynak
Abstract:
The method of occupation kernels has been used to learn ordinary differential equations from data in a non-parametric way. We propose a two-step method for learning the drift and diffusion of a stochastic differential equation given snapshots of the process. In the first step, we learn the drift by applying the occupation kernel algorithm to the expected value of the process. In the second step, w…
▽ More
The method of occupation kernels has been used to learn ordinary differential equations from data in a non-parametric way. We propose a two-step method for learning the drift and diffusion of a stochastic differential equation given snapshots of the process. In the first step, we learn the drift by applying the occupation kernel algorithm to the expected value of the process. In the second step, we learn the diffusion given the drift using a semi-definite program. Specifically, we learn the diffusion squared as a non-negative function in a RKHS associated with the square of a kernel. We present examples and simulations.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
A Finite-Horizon Approach to Active Level Set Estimation
Authors:
Phillip Kearns,
Bruno Jedynak,
John Lipor
Abstract:
We consider the problem of active learning in the context of spatial sampling for level set estimation (LSE), where the goal is to localize all regions where a function of interest lies above/below a given threshold as quickly as possible. We present a finite-horizon search procedure to perform LSE in one dimension while optimally balancing both the final estimation error and the distance traveled…
▽ More
We consider the problem of active learning in the context of spatial sampling for level set estimation (LSE), where the goal is to localize all regions where a function of interest lies above/below a given threshold as quickly as possible. We present a finite-horizon search procedure to perform LSE in one dimension while optimally balancing both the final estimation error and the distance traveled for a fixed number of samples. A tuning parameter is used to trade off between the estimation accuracy and distance traveled. We show that the resulting optimization problem can be solved in closed form and that the resulting policy generalizes existing approaches to this problem. We then show how this approach can be used to perform level set estimation in higher dimensions under the popular Gaussian process model. Empirical results on synthetic data indicate that as the cost of travel increases, our method's ability to treat distance nonmyopically allows it to significantly improve on the state of the art. On real air quality data, our approach achieves roughly one fifth the estimation error at less than half the cost of competing algorithms.
△ Less
Submitted 18 October, 2023;
originally announced October 2023.
-
Learning High-Dimensional Nonparametric Differential Equations via Multivariate Occupation Kernel Functions
Authors:
Victor Rielly,
Kamel Lahouel,
Ethan Lew,
Michael Wells,
Vicky Haney,
Bruno Jedynak
Abstract:
Learning a nonparametric system of ordinary differential equations (ODEs) from $n$ trajectory snapshots in a $d$-dimensional state space requires learning $d$ functions of $d$ variables. Explicit formulations scale quadratically in $d$ unless additional knowledge about system properties, such as sparsity and symmetries, is available. In this work, we propose a linear approach to learning using the…
▽ More
Learning a nonparametric system of ordinary differential equations (ODEs) from $n$ trajectory snapshots in a $d$-dimensional state space requires learning $d$ functions of $d$ variables. Explicit formulations scale quadratically in $d$ unless additional knowledge about system properties, such as sparsity and symmetries, is available. In this work, we propose a linear approach to learning using the implicit formulation provided by vector-valued Reproducing Kernel Hilbert Spaces. By rewriting the ODEs in a weaker integral form, which we subsequently minimize, we derive our learning algorithm. The minimization problem's solution for the vector field relies on multivariate occupation kernel functions associated with the solution trajectories. We validate our approach through experiments on highly nonlinear simulated and real data, where $d$ may exceed 100. We further demonstrate the versatility of the proposed method by learning a nonparametric first order quasilinear partial differential equation.
△ Less
Submitted 16 June, 2023;
originally announced June 2023.
-
Learning nonparametric ordinary differential equations from noisy data
Authors:
Kamel Lahouel,
Michael Wells,
Victor Rielly,
Ethan Lew,
David Lovitz,
Bruno M. Jedynak
Abstract:
Learning nonparametric systems of Ordinary Differential Equations (ODEs) dot x = f(t,x) from noisy data is an emerging machine learning topic. We use the well-developed theory of Reproducing Kernel Hilbert Spaces (RKHS) to define candidates for f for which the solution of the ODE exists and is unique. Learning f consists of solving a constrained optimization problem in an RKHS. We propose a penalt…
▽ More
Learning nonparametric systems of Ordinary Differential Equations (ODEs) dot x = f(t,x) from noisy data is an emerging machine learning topic. We use the well-developed theory of Reproducing Kernel Hilbert Spaces (RKHS) to define candidates for f for which the solution of the ODE exists and is unique. Learning f consists of solving a constrained optimization problem in an RKHS. We propose a penalty method that iteratively uses the Representer theorem and Euler approximations to provide a numerical solution. We prove a generalization bound for the L2 distance between x and its estimator and provide experimental comparisons with the state-of-the-art.
△ Less
Submitted 12 November, 2023; v1 submitted 30 June, 2022;
originally announced June 2022.
-
Unstructured Primary Outcome in Randomized Controlled Trials
Authors:
Daniel Taylor-Rodriguez,
David Lovitz,
Nora Mattek,
Chao-Yi Wu,
Hiroko Dodge,
Jeffrey Kaye,
Bruno M. Jedynak
Abstract:
The primary outcome of Randomized clinical Trials (RCTs) are typically dichotomous, continuous, multivariate continuous, or time-to-event. However, what if this outcome is unstructured, e.g., a list of variables of mixed types, longitudinal sequences, images, audio recordings, etc. When the outcome is unstructured it is unclear how to assess RCT success and how to compute sample size. We show that…
▽ More
The primary outcome of Randomized clinical Trials (RCTs) are typically dichotomous, continuous, multivariate continuous, or time-to-event. However, what if this outcome is unstructured, e.g., a list of variables of mixed types, longitudinal sequences, images, audio recordings, etc. When the outcome is unstructured it is unclear how to assess RCT success and how to compute sample size. We show that kernel methods offer natural extensions to traditional biostatistics methods. We demonstrate our approach with the measurements of computer usage in a cohort of aging participants, some of which will become cognitively impaired. Simulations as well as a real data experiment show the superiority of the proposed approach compared to the standard in this situation: generalized mixed effect models.
△ Less
Submitted 25 November, 2020;
originally announced November 2020.
-
The Alzheimer's Disease Prediction Of Longitudinal Evolution (TADPOLE) Challenge: Results after 1 Year Follow-up
Authors:
Razvan V. Marinescu,
Neil P. Oxtoby,
Alexandra L. Young,
Esther E. Bron,
Arthur W. Toga,
Michael W. Weiner,
Frederik Barkhof,
Nick C. Fox,
Arman Eshaghi,
Tina Toni,
Marcin Salaterski,
Veronika Lunina,
Manon Ansart,
Stanley Durrleman,
Pascal Lu,
Samuel Iddi,
Dan Li,
Wesley K. Thompson,
Michael C. Donohue,
Aviv Nahon,
Yarden Levy,
Dan Halbersberg,
Mariya Cohen,
Huiling Liao,
Tengfei Li
, et al. (71 additional authors not shown)
Abstract:
We present the findings of "The Alzheimer's Disease Prediction Of Longitudinal Evolution" (TADPOLE) Challenge, which compared the performance of 92 algorithms from 33 international teams at predicting the future trajectory of 219 individuals at risk of Alzheimer's disease. Challenge participants were required to make a prediction, for each month of a 5-year future time period, of three key outcome…
▽ More
We present the findings of "The Alzheimer's Disease Prediction Of Longitudinal Evolution" (TADPOLE) Challenge, which compared the performance of 92 algorithms from 33 international teams at predicting the future trajectory of 219 individuals at risk of Alzheimer's disease. Challenge participants were required to make a prediction, for each month of a 5-year future time period, of three key outcomes: clinical diagnosis, Alzheimer's Disease Assessment Scale Cognitive Subdomain (ADAS-Cog13), and total volume of the ventricles. The methods used by challenge participants included multivariate linear regression, machine learning methods such as support vector machines and deep neural networks, as well as disease progression models. No single submission was best at predicting all three outcomes. For clinical diagnosis and ventricle volume prediction, the best algorithms strongly outperform simple baselines in predictive ability. However, for ADAS-Cog13 no single submitted prediction method was significantly better than random guesswork. Two ensemble methods based on taking the mean and median over all predictions, obtained top scores on almost all tasks. Better than average performance at diagnosis prediction was generally associated with the additional inclusion of features from cerebrospinal fluid (CSF) samples and diffusion tensor imaging (DTI). On the other hand, better performance at ventricle volume prediction was associated with inclusion of summary statistics, such as the slope or maxima/minima of biomarkers. TADPOLE's unique results suggest that current prediction algorithms provide sufficient accuracy to exploit biomarkers related to clinical diagnosis and ventricle volume, for cohort refinement in clinical trials for Alzheimer's disease. However, results call into question the usage of cognitive test scores for patient selection and as a primary endpoint in clinical trials.
△ Less
Submitted 27 December, 2021; v1 submitted 9 February, 2020;
originally announced February 2020.
-
Machine Learning for Health (ML4H) Workshop at NeurIPS 2018
Authors:
Natalia Antropova,
Andrew L. Beam,
Brett K. Beaulieu-Jones,
Irene Chen,
Corey Chivers,
Adrian Dalca,
Sam Finlayson,
Madalina Fiterau,
Jason Alan Fries,
Marzyeh Ghassemi,
Mike Hughes,
Bruno Jedynak,
Jasvinder S. Kandola,
Matthew McDermott,
Tristan Naumann,
Peter Schulam,
Farah Shamout,
Alexandre Yahi
Abstract:
This volume represents the accepted submissions from the Machine Learning for Health (ML4H) workshop at the conference on Neural Information Processing Systems (NeurIPS) 2018, held on December 8, 2018 in Montreal, Canada.
This volume represents the accepted submissions from the Machine Learning for Health (ML4H) workshop at the conference on Neural Information Processing Systems (NeurIPS) 2018, held on December 8, 2018 in Montreal, Canada.
△ Less
Submitted 24 November, 2018; v1 submitted 17 November, 2018;
originally announced November 2018.
-
Topology guaranteed segmentation of the human retina from OCT using convolutional neural networks
Authors:
Yufan He,
Aaron Carass,
Bruno M. Jedynak,
Sharon D. Solomon,
Shiv Saidha,
Peter A. Calabresi,
Jerry L. Prince
Abstract:
Optical coherence tomography (OCT) is a noninvasive imaging modality which can be used to obtain depth images of the retina. The changing layer thicknesses can thus be quantified by analyzing these OCT images, moreover these changes have been shown to correlate with disease progression in multiple sclerosis. Recent automated retinal layer segmentation tools use machine learning methods to perform…
▽ More
Optical coherence tomography (OCT) is a noninvasive imaging modality which can be used to obtain depth images of the retina. The changing layer thicknesses can thus be quantified by analyzing these OCT images, moreover these changes have been shown to correlate with disease progression in multiple sclerosis. Recent automated retinal layer segmentation tools use machine learning methods to perform pixel-wise labeling and graph methods to guarantee the layer hierarchy or topology. However, graph parameters like distance and smoothness constraints must be experimentally assigned by retinal region and pathology, thus degrading the flexibility and time efficiency of the whole framework. In this paper, we develop cascaded deep networks to provide a topologically correct segmentation of the retinal layers in a single feed forward propagation. The first network (S-Net) performs pixel-wise labeling and the second regression network (R-Net) takes the topologically unconstrained S-Net results and outputs layer thicknesses for each layer and each position. Relu activation is used as the final operation of the R-Net which guarantees non-negativity of the output layer thickness. Since the segmentation boundary position is acquired by summing up the corresponding non-negative layer thicknesses, the layer ordering (i.e., topology) of the reconstructed boundaries is guaranteed even at the fovea where the distances between boundaries can be zero. The R-Net is trained using simulated masks and thus can be generalized to provide topology guaranteed segmentation for other layered structures. This deep network has achieved comparable mean absolute boundary error (2.82 μm) to state-of-the-art graph methods (2.83 μm).
△ Less
Submitted 13 March, 2018;
originally announced March 2018.
-
Gaussian Processes with Context-Supported Priors for Active Object Localization
Authors:
Anthony D. Rhodes,
Jordan Witte,
Melanie Mitchell,
Bruno Jedynak
Abstract:
We devise an algorithm using a Bayesian optimization framework in conjunction with contextual visual data for the efficient localization of objects in still images. Recent research has demonstrated substantial progress in object localization and related tasks for computer vision. However, many current state-of-the-art object localization procedures still suffer from inaccuracy and inefficiency, in…
▽ More
We devise an algorithm using a Bayesian optimization framework in conjunction with contextual visual data for the efficient localization of objects in still images. Recent research has demonstrated substantial progress in object localization and related tasks for computer vision. However, many current state-of-the-art object localization procedures still suffer from inaccuracy and inefficiency, in addition to failing to provide a principled and interpretable system amenable to high-level vision tasks. We address these issues with the current research.
Our method encompasses an active search procedure that uses contextual data to generate initial bounding-box proposals for a target object. We train a convolutional neural network to approximate an offset distance from the target object. Next, we use a Gaussian Process to model this offset response signal over the search space of the target. We then employ a Bayesian active search for accurate localization of the target.
In experiments, we compare our approach to a state-of-theart bounding-box regression method for a challenging pedestrian localization task. Our method exhibits a substantial improvement over this baseline regression method.
△ Less
Submitted 20 September, 2017; v1 submitted 25 March, 2017;
originally announced March 2017.
-
A multivariate nonlinear mixed effects model for longitudinal image analysis: Application to amyloid imaging
Authors:
Murat Bilgel,
Jerry L. Prince,
Dean F. Wong,
Susan M. Resnick,
Bruno M. Jedynak
Abstract:
It is important to characterize the temporal trajectories of disease-related biomarkers in order to monitor progression and identify potential points of intervention. This is especially important for neurodegenerative diseases, as therapeutic intervention is most likely to be effective in the preclinical disease stages prior to significant neuronal damage. Longitudinal neuroimaging allows for the…
▽ More
It is important to characterize the temporal trajectories of disease-related biomarkers in order to monitor progression and identify potential points of intervention. This is especially important for neurodegenerative diseases, as therapeutic intervention is most likely to be effective in the preclinical disease stages prior to significant neuronal damage. Longitudinal neuroimaging allows for the measurement of structural, functional, and metabolic integrity of the brain over time at the level of voxels. However, commonly used longitudinal analysis approaches, such as linear mixed effects models, do not account for the fact that individuals enter a study at various disease stages and progress at different rates, and generally consider each voxelwise measure independently. We propose a multivariate nonlinear mixed effects model for estimating the trajectories of voxelwise neuroimaging biomarkers from longitudinal data that accounts for such differences across individuals. The method involves the prediction of a progression score for each visit based on a collective analysis of voxelwise biomarker data within an expectation-maximization framework that efficiently handles large amounts of measurements and variable number of visits per individual, and accounts for spatial correlations among voxels. This score allows individuals with similar progressions to be aligned and analyzed together, which enables the construction of a trajectory of brain changes as a function of an underlying progression or disease stage. Application of our method to studying images of beta-amyloid deposition, a hallmark of preclinical Alzheimer's disease, suggests that precuneus is the earliest cortical region to accumulate amyloid. The proposed method can be applied to other types of longitudinal imaging data, including metabolism, blood flow, tau, and structural imaging-derived measures.
△ Less
Submitted 4 April, 2016;
originally announced April 2016.
-
Non-Adaptive Policies for 20 Questions Target Localization
Authors:
Ehsan Variani,
Kamel Lahouel,
Avner Bar-Hen,
Bruno Jedynak
Abstract:
The problem of target localization with noise is addressed. The target is a sample from a continuous random variable with known distribution and the goal is to locate it with minimum mean squared error distortion. The localization scheme or policy proceeds by queries, or questions, weather or not the target belongs to some subset as it is addressed in the 20-question framework. These subsets are n…
▽ More
The problem of target localization with noise is addressed. The target is a sample from a continuous random variable with known distribution and the goal is to locate it with minimum mean squared error distortion. The localization scheme or policy proceeds by queries, or questions, weather or not the target belongs to some subset as it is addressed in the 20-question framework. These subsets are not constrained to be intervals and the answers to the queries are noisy. While this situation is well studied for adaptive querying, this paper is focused on the non adaptive querying policies based on dyadic questions. The asymptotic minimum achievable distortion under such policies is derived. Furthermore, a policy named the Aurelian1 is exhibited which achieves asymptotically this distortion.
△ Less
Submitted 1 May, 2015; v1 submitted 22 April, 2015;
originally announced April 2015.
-
Probabilistic Group Testing under Sum Observations: A Parallelizable 2-Approximation for Entropy Loss
Authors:
Weidong Han,
Purnima Rajan,
Peter I. Frazier,
Bruno M. Jedynak
Abstract:
We consider the problem of group testing with sum observations and noiseless answers, in which we aim to locate multiple objects by querying the number of objects in each of a sequence of chosen sets. We study a probabilistic setting with entropy loss, in which we assume a joint Bayesian prior density on the locations of the objects and seek to choose the sets queried to minimize the expected entr…
▽ More
We consider the problem of group testing with sum observations and noiseless answers, in which we aim to locate multiple objects by querying the number of objects in each of a sequence of chosen sets. We study a probabilistic setting with entropy loss, in which we assume a joint Bayesian prior density on the locations of the objects and seek to choose the sets queried to minimize the expected entropy of the Bayesian posterior distribution after a fixed number of questions. We present a new non-adaptive policy, called the dyadic policy, show it is optimal among non-adaptive policies, and is within a factor of two of optimal among adaptive policies. This policy is quick to compute, its nonadaptive nature makes it easy to parallelize, and our bounds show it performs well even when compared with adaptive policies. We also study an adaptive greedy policy, which maximizes the one-step expected reduction in entropy, and show that it performs at least as well as the dyadic policy, offering greater query efficiency but reduced parallelism. Numerical experiments demonstrate that both procedures outperform a divide-and-conquer benchmark policy from the literature, called sequential bifurcation, and show how these procedures may be applied in a stylized computer vision problem.
△ Less
Submitted 22 September, 2015; v1 submitted 16 July, 2014;
originally announced July 2014.
-
Active Testing for Face Detection and Localization
Authors:
Raphael Sznitman,
Bruno Jedynak
Abstract:
We provide a novel search technique, which uses a hierarchical model and a mutual information gain heuristic to efficiently prune the search space when localizing faces in images. We show exponential gains in computation over traditional sliding window approaches, while kee** similar performance levels.
We provide a novel search technique, which uses a hierarchical model and a mutual information gain heuristic to efficiently prune the search space when localizing faces in images. We show exponential gains in computation over traditional sliding window approaches, while kee** similar performance levels.
△ Less
Submitted 26 March, 2010;
originally announced March 2010.
