-
Data-efficient operator learning for solving high Mach number fluid flow problems
Authors:
Noah Ford,
Victor J. Leon,
Honest Mrema,
Jeffrey Gilbert,
Alexander New
Abstract:
We consider the problem of using SciML to predict solutions of high Mach fluid flows over irregular geometries. In this setting, data is limited, and so it is desirable for models to perform well in the low-data setting. We show that Neural Basis Functions (NBF), which learns a basis of behavior modes from the data and then uses this basis to make predictions, is more effective than a basis-unawar…
▽ More
We consider the problem of using SciML to predict solutions of high Mach fluid flows over irregular geometries. In this setting, data is limited, and so it is desirable for models to perform well in the low-data setting. We show that Neural Basis Functions (NBF), which learns a basis of behavior modes from the data and then uses this basis to make predictions, is more effective than a basis-unaware baseline model. In addition, we identify continuing challenges in the space of predicting solutions for this type of problem.
△ Less
Submitted 4 December, 2023; v1 submitted 28 November, 2023;
originally announced November 2023.
-
Ensemble models outperform single model uncertainties and predictions for operator-learning of hypersonic flows
Authors:
Victor J. Leon,
Noah Ford,
Honest Mrema,
Jeffrey Gilbert,
Alexander New
Abstract:
High-fidelity computational simulations and physical experiments of hypersonic flows are resource intensive. Training scientific machine learning (SciML) models on limited high-fidelity data offers one approach to rapidly predict behaviors for situations that have not been seen before. However, high-fidelity data is itself in limited quantity to validate all outputs of the SciML model in unexplore…
▽ More
High-fidelity computational simulations and physical experiments of hypersonic flows are resource intensive. Training scientific machine learning (SciML) models on limited high-fidelity data offers one approach to rapidly predict behaviors for situations that have not been seen before. However, high-fidelity data is itself in limited quantity to validate all outputs of the SciML model in unexplored input space. As such, an uncertainty-aware SciML model is desired. The SciML model's output uncertainties could then be used to assess the reliability and confidence of the model's predictions. In this study, we extend a DeepONet using three different uncertainty quantification mechanisms: mean-variance estimation, evidential uncertainty, and ensembling. The uncertainty aware DeepONet models are trained and evaluated on the hypersonic flow around a blunt cone object with data generated via computational fluid dynamics over a wide range of Mach numbers and altitudes. We find that ensembling outperforms the other two uncertainty models in terms of minimizing error and calibrating uncertainty in both interpolative and extrapolative regimes.
△ Less
Submitted 3 November, 2023; v1 submitted 31 October, 2023;
originally announced November 2023.
-
DeepFDR: A Deep Learning-based False Discovery Rate Control Method for Neuroimaging Data
Authors:
Taehyo Kim,
Hai Shu,
Qiran Jia,
Mony J. de Leon
Abstract:
Voxel-based multiple testing is widely used in neuroimaging data analysis. Traditional false discovery rate (FDR) control methods often ignore the spatial dependence among the voxel-based tests and thus suffer from substantial loss of testing power. While recent spatial FDR control methods have emerged, their validity and optimality remain questionable when handling the complex spatial dependencie…
▽ More
Voxel-based multiple testing is widely used in neuroimaging data analysis. Traditional false discovery rate (FDR) control methods often ignore the spatial dependence among the voxel-based tests and thus suffer from substantial loss of testing power. While recent spatial FDR control methods have emerged, their validity and optimality remain questionable when handling the complex spatial dependencies of the brain. Concurrently, deep learning methods have revolutionized image segmentation, a task closely related to voxel-based multiple testing. In this paper, we propose DeepFDR, a novel spatial FDR control method that leverages unsupervised deep learning-based image segmentation to address the voxel-based multiple testing problem. Numerical studies, including comprehensive simulations and Alzheimer's disease FDG-PET image analysis, demonstrate DeepFDR's superiority over existing methods. DeepFDR not only excels in FDR control and effectively diminishes the false nondiscovery rate, but also boasts exceptional computational efficiency highly suited for tackling large-scale neuroimaging data.
△ Less
Submitted 10 March, 2024; v1 submitted 20 October, 2023;
originally announced October 2023.
-
Efficient Computation of Shap Explanation Scores for Neural Network Classifiers via Knowledge Compilation
Authors:
Leopoldo Bertossi,
Jorge E. Leon
Abstract:
The use of Shap scores has become widespread in Explainable AI. However, their computation is in general intractable, in particular when done with a black-box classifier, such as neural network. Recent research has unveiled classes of open-box Boolean Circuit classifiers for which Shap can be computed efficiently. We show how to transform binary neural networks into those circuits for efficient Sh…
▽ More
The use of Shap scores has become widespread in Explainable AI. However, their computation is in general intractable, in particular when done with a black-box classifier, such as neural network. Recent research has unveiled classes of open-box Boolean Circuit classifiers for which Shap can be computed efficiently. We show how to transform binary neural networks into those circuits for efficient Shap computation.We use logic-based knowledge compilation techniques. The performance gain is huge, as we show in the light of our experiments.
△ Less
Submitted 22 July, 2023; v1 submitted 11 March, 2023;
originally announced March 2023.
-
Intuitive and Efficient Human-robot Collaboration via Real-time Approximate Bayesian Inference
Authors:
Javier Felip Leon,
David Gonzalez-Aguirre,
Lama Nachman
Abstract:
The combination of collaborative robots and end-to-end AI, promises flexible automation of human tasks in factories and warehouses. However, such promise seems a few breakthroughs away. In the meantime, humans and cobots will collaborate hel** each other. For these collaborations to be effective and safe, robots need to model, predict and exploit human's intents for responsive decision making pr…
▽ More
The combination of collaborative robots and end-to-end AI, promises flexible automation of human tasks in factories and warehouses. However, such promise seems a few breakthroughs away. In the meantime, humans and cobots will collaborate hel** each other. For these collaborations to be effective and safe, robots need to model, predict and exploit human's intents for responsive decision making processes.
Approximate Bayesian Computation (ABC) is an analysis-by-synthesis approach to perform probabilistic predictions upon uncertain quantities. ABC includes priors conveniently, leverages sampling algorithms for inference and is flexible to benefit from complex models, e.g. via simulators. However, ABC is known to be computationally too intensive to run at interactive frame rates required for effective human-robot collaboration tasks.
In this paper, we formulate human reaching intent prediction as an ABC problem and describe two key performance innovations which allow computations at interactive rates. Our real-world experiments with a collaborative robot set-up, demonstrate the viability of our proposed approach. Experimental evaluations convey the advantages and value of human intent prediction for packing cooperative tasks. Qualitative results show how anticipating human's reaching intent improves human-robot collaboration without compromising safety. Quantitative task fluency metrics confirm the qualitative claims.
△ Less
Submitted 17 May, 2022;
originally announced May 2022.
-
Depth Pruning with Auxiliary Networks for TinyML
Authors:
Josen Daniel De Leon,
Rowel Atienza
Abstract:
Pruning is a neural network optimization technique that sacrifices accuracy in exchange for lower computational requirements. Pruning has been useful when working with extremely constrained environments in tinyML. Unfortunately, special hardware requirements and limited study on its effectiveness on already compact models prevent its wider adoption. Depth pruning is a form of pruning that requires…
▽ More
Pruning is a neural network optimization technique that sacrifices accuracy in exchange for lower computational requirements. Pruning has been useful when working with extremely constrained environments in tinyML. Unfortunately, special hardware requirements and limited study on its effectiveness on already compact models prevent its wider adoption. Depth pruning is a form of pruning that requires no specialized hardware but suffers from a large accuracy falloff. To improve this, we propose a modification that utilizes a highly efficient auxiliary network as an effective interpreter of intermediate feature maps. Our results show a parameter reduction of 93% on the MLPerfTiny Visual Wakewords (VWW) task and 28% on the Keyword Spotting (KWS) task with accuracy cost of 0.65% and 1.06% respectively. When evaluated on a Cortex-M0 microcontroller, our proposed method reduces the VWW model size by 4.7x and latency by 1.6x while counter intuitively gaining 1% accuracy. KWS model size on Cortex-M0 was also reduced by 1.2x and latency by 1.2x at the cost of 2.21% accuracy.
△ Less
Submitted 22 April, 2022;
originally announced April 2022.
-
Towards Label-Free 3D Segmentation of Optical Coherence Tomography Images of the Optic Nerve Head Using Deep Learning
Authors:
Sripad Krishna Devalla,
Tan Hung Pham,
Satish Kumar Panda,
Liang Zhang,
Giridhar Subramanian,
Anirudh Swaminathan,
Chin Zhi Yun,
Mohan Rajan,
Sujatha Mohan,
Ramaswami Krishnadas,
Vijayalakshmi Senthil,
John Mark S. de Leon,
Tin A. Tun,
Ching-Yu Cheng,
Leopold Schmetterer,
Shamira Perera,
Tin Aung,
Alexandre H. Thiery,
Michael J. A. Girard
Abstract:
Since the introduction of optical coherence tomography (OCT), it has been possible to study the complex 3D morphological changes of the optic nerve head (ONH) tissues that occur along with the progression of glaucoma. Although several deep learning (DL) techniques have been recently proposed for the automated extraction (segmentation) and quantification of these morphological changes, the device s…
▽ More
Since the introduction of optical coherence tomography (OCT), it has been possible to study the complex 3D morphological changes of the optic nerve head (ONH) tissues that occur along with the progression of glaucoma. Although several deep learning (DL) techniques have been recently proposed for the automated extraction (segmentation) and quantification of these morphological changes, the device specific nature and the difficulty in preparing manual segmentations (training data) limit their clinical adoption. With several new manufacturers and next-generation OCT devices entering the market, the complexity in deploying DL algorithms clinically is only increasing. To address this, we propose a DL based 3D segmentation framework that is easily translatable across OCT devices in a label-free manner (i.e. without the need to manually re-segment data for each device). Specifically, we developed 2 sets of DL networks. The first (referred to as the enhancer) was able to enhance OCT image quality from 3 OCT devices, and harmonized image-characteristics across these devices. The second performed 3D segmentation of 6 important ONH tissue layers. We found that the use of the enhancer was critical for our segmentation network to achieve device independency. In other words, our 3D segmentation network trained on any of 3 devices successfully segmented ONH tissue layers from the other two devices with high performance (Dice coefficients > 0.92). With such an approach, we could automatically segment images from new OCT devices without ever needing manual segmentation data from such devices.
△ Less
Submitted 22 February, 2020;
originally announced February 2020.
-
On finite-time and fixed-time consensus algorithms for dynamic networks switching among disconnected digraphs
Authors:
David Gómez-Gutiérrez,
Carlos Renato Vázquez,
Sergej Čelikovský,
Juan Diego Sánchez-Torres,
Javier Ruiz León
Abstract:
The aim of this paper is to analyze a class of consensus algorithms with finite-time or fixed-time convergence for dynamic networks formed by agents with first-order dynamics. In particular, in the analyzed class a single evaluation of a nonlinear function of the consensus error is performed per each node. The classical assumption of switching among connected graphs is dropped here, allowing to re…
▽ More
The aim of this paper is to analyze a class of consensus algorithms with finite-time or fixed-time convergence for dynamic networks formed by agents with first-order dynamics. In particular, in the analyzed class a single evaluation of a nonlinear function of the consensus error is performed per each node. The classical assumption of switching among connected graphs is dropped here, allowing to represent failures and intermittent communications between agents. Thus, conditions to guarantee finite and fixed-time convergence, even while switching among disconnected graphs, are provided. Moreover, the algorithms of the considered class are shown to be computationally simpler than previously proposed finite-time consensus algorithms for dynamic networks, which is an important feature in scenarios with computationally limited nodes and energy efficiency requirements such as in sensor networks. The performance of the considered consensus algorithms is illustrated through simulations, comparing it to existing approaches for dynamic networks with finite-time and fixed-time convergence. It is shown that the settling time of the considered algorithms grows slower when the number of nodes increases than with other consensus algorithms for dynamic networks.
△ Less
Submitted 25 June, 2021; v1 submitted 31 October, 2018;
originally announced November 2018.
-
Raspberry Pi and Arduino Uno Working together as a Basic Meteorological Station
Authors:
José Rafael Cortés León,
Ricardo Francisco Martínez-González,
Anilú Miranda Medina,
Luis Alberto Peralta-Pelaez
Abstract:
The present paper describes a novel Raspberry Pi and Arduino UNO architecture used as a meteorological station. One of the advantages of the proposed architecture is the huge quantity of sensors developed for its usage; practically one can find them for any application, and weather sensing is not an exception. The principle followed is to configure Raspberry as a collector for measures obtained fr…
▽ More
The present paper describes a novel Raspberry Pi and Arduino UNO architecture used as a meteorological station. One of the advantages of the proposed architecture is the huge quantity of sensors developed for its usage; practically one can find them for any application, and weather sensing is not an exception. The principle followed is to configure Raspberry as a collector for measures obtained from Arduino, transmitting occurs via USB; meanwhile, Raspberry broadcasts them via a web page. For such activity is possible thanks to Raspbian, a Linux-based operating system. It has a lot of libraries and resources available, among them Apache Web Server, that gives the possibility to host a web-page. On it, the user can observe temperature, humidity, solar radiance, and wind speed and direction. Information on the web-page is refreshed each five minute; however, measurements arrive at Raspberry every ten seconds. This low refreshment rate was determined because weather variables normally do not abruptly change. As an additional feature, system stores all information on the log file, this gives the possibility for future analysis and processing.
△ Less
Submitted 21 November, 2017;
originally announced November 2017.
-
Multi-View Dynamic Facial Action Unit Detection
Authors:
Andres Romero,
Juan Leon,
Pablo Arbelaez
Abstract:
We propose a novel convolutional neural network approach to address the fine-grained recognition problem of multi-view dynamic facial action unit detection. We leverage recent gains in large-scale object recognition by formulating the task of predicting the presence or absence of a specific action unit in a still image of a human face as holistic classification. We then explore the design space of…
▽ More
We propose a novel convolutional neural network approach to address the fine-grained recognition problem of multi-view dynamic facial action unit detection. We leverage recent gains in large-scale object recognition by formulating the task of predicting the presence or absence of a specific action unit in a still image of a human face as holistic classification. We then explore the design space of our approach by considering both shared and independent representations for separate action units, and also different CNN architectures for combining color and motion information. We then move to the novel setup of the FERA 2017 Challenge, in which we propose a multi-view extension of our approach that operates by first predicting the viewpoint from which the video was taken, and then evaluating an ensemble of action unit detectors that were trained for that specific viewpoint. Our approach is holistic, efficient, and modular, since new action units can be easily included in the overall system. Our approach significantly outperforms the baseline of the FERA 2017 Challenge, with an absolute improvement of 14% on the F1-metric. Additionally, it compares favorably against the winner of the FERA 2017 challenge. Code source is available at https://github.com/BCV-Uniandes/AUNets.
△ Less
Submitted 20 August, 2018; v1 submitted 25 April, 2017;
originally announced April 2017.
-
Latent Dirichlet Allocation Uncovers Spectral Characteristics of Drought Stressed Plants
Authors:
Mirwaes Wahabzada,
Kristian Kersting,
Christian Bauckhage,
Christoph Roemer,
Agim Ballvora,
Francisco Pinto,
Uwe Rascher,
Jens Leon,
Lutz Ploemer
Abstract:
Understanding the adaptation process of plants to drought stress is essential in improving management practices, breeding strategies as well as engineering viable crops for a sustainable agriculture in the coming decades. Hyper-spectral imaging provides a particularly promising approach to gain such understanding since it allows to discover non-destructively spectral characteristics of plants gove…
▽ More
Understanding the adaptation process of plants to drought stress is essential in improving management practices, breeding strategies as well as engineering viable crops for a sustainable agriculture in the coming decades. Hyper-spectral imaging provides a particularly promising approach to gain such understanding since it allows to discover non-destructively spectral characteristics of plants governed primarily by scattering and absorption characteristics of the leaf internal structure and biochemical constituents. Several drought stress indices have been derived using hyper-spectral imaging. However, they are typically based on few hyper-spectral images only, rely on interpretations of experts, and consider few wavelengths only. In this study, we present the first data-driven approach to discovering spectral drought stress indices, treating it as an unsupervised labeling problem at massive scale. To make use of short range dependencies of spectral wavelengths, we develop an online variational Bayes algorithm for latent Dirichlet allocation with convolved Dirichlet regularizer. This approach scales to massive datasets and, hence, provides a more objective complement to plant physiological practices. The spectral topics found conform to plant physiological knowledge and can be computed in a fraction of the time compared to existing LDA approaches.
△ Less
Submitted 16 October, 2012;
originally announced October 2012.
-
Filling-Based Techniques Applied to Object Projection Feature Estimation
Authors:
Luis Quesada,
Alejandro J. León
Abstract:
3D motion tracking is a critical task in many computer vision applications. Unsupervised markerless 3D motion tracking systems determine the most relevant object in the screen and then track it by continuously estimating its projection features (center and area) from the edge image and a point inside the relevant object projection (namely, inner point), until the tracking fails. Existing object pr…
▽ More
3D motion tracking is a critical task in many computer vision applications. Unsupervised markerless 3D motion tracking systems determine the most relevant object in the screen and then track it by continuously estimating its projection features (center and area) from the edge image and a point inside the relevant object projection (namely, inner point), until the tracking fails. Existing object projection feature estimation techniques are based on ray-casting from the inner point. These techniques present three main drawbacks: when the inner point is surrounded by edges, rays may not reach other relevant areas; as a consequence of that issue, the estimated features may greatly vary depending on the position of the inner point relative to the object projection; and finally, increasing the number of rays being casted and the ray-casting iterations (which would make the results more accurate and stable) increases the processing time to the point the tracking cannot be performed on the fly. In this paper, we analyze an intuitive filling-based object projection feature estimation technique that solves the aforementioned problems but is too sensitive to edge miscalculations. Then, we propose a less computing-intensive modification to that technique that would not be affected by the existing techniques issues and would be no more sensitive to edge miscalculations than ray-casting-based techniques.
△ Less
Submitted 29 February, 2012;
originally announced February 2012.
-
The Object Projection Feature Estimation Problem in Unsupervised Markerless 3D Motion Tracking
Authors:
Luis Quesada,
Alejandro J. León
Abstract:
3D motion tracking is a critical task in many computer vision applications. Existing 3D motion tracking techniques require either a great amount of knowledge on the target object or specific hardware. These requirements discourage the wide spread of commercial applications based on 3D motion tracking. 3D motion tracking systems that require no knowledge on the target object and run on a single low…
▽ More
3D motion tracking is a critical task in many computer vision applications. Existing 3D motion tracking techniques require either a great amount of knowledge on the target object or specific hardware. These requirements discourage the wide spread of commercial applications based on 3D motion tracking. 3D motion tracking systems that require no knowledge on the target object and run on a single low-budget camera require estimations of the object projection features (namely, area and position). In this paper, we define the object projection feature estimation problem and we present a novel 3D motion tracking system that needs no knowledge on the target object and that only requires a single low-budget camera, as installed in most computers and smartphones. Our system estimates, in real time, the three-dimensional position of a non-modeled unmarked object that may be non-rigid, non-convex, partially occluded, self occluded, or motion blurred, given that it is opaque, evenly colored, and enough contrasting with the background in each frame. Our system is also able to determine the most relevant object to track in the screen. Our 3D motion tracking system does not impose hard constraints, therefore it allows a market-wide implementation of applications that use 3D motion tracking.
△ Less
Submitted 18 November, 2011; v1 submitted 16 November, 2011;
originally announced November 2011.
