Showing 1–29 of 29 results for author: Rousseau, F

Hidden Flaws Behind Expert-Level Accuracy of GPT-4 Vision in Medicine

Authors: Qiao **, Fangyuan Chen, Yiliang Zhou, Ziyang Xu, Justin M. Cheung, Robert Chen, Ronald M. Summers, Justin F. Rousseau, Peiyun Ni, Marc J Landsman, Sally L. Baxter, Subhi J. Al'Aref, Yijia Li, Alex Chen, Josef A. Brejt, Michael F. Chiang, Yifan Peng, Zhiyong Lu

Abstract: Recent studies indicate that Generative Pre-trained Transformer 4 with Vision (GPT-4V) outperforms human physicians in medical challenge tasks. However, these evaluations primarily focused on the accuracy of multi-choice questions alone. Our study extends the current scope by conducting a comprehensive analysis of GPT-4V's rationales of image comprehension, recall of medical knowledge, and step-by… ▽ More Recent studies indicate that Generative Pre-trained Transformer 4 with Vision (GPT-4V) outperforms human physicians in medical challenge tasks. However, these evaluations primarily focused on the accuracy of multi-choice questions alone. Our study extends the current scope by conducting a comprehensive analysis of GPT-4V's rationales of image comprehension, recall of medical knowledge, and step-by-step multimodal reasoning when solving New England Journal of Medicine (NEJM) Image Challenges - an imaging quiz designed to test the knowledge and diagnostic capabilities of medical professionals. Evaluation results confirmed that GPT-4V performs comparatively to human physicians regarding multi-choice accuracy (81.6% vs. 77.8%). GPT-4V also performs well in cases where physicians incorrectly answer, with over 78% accuracy. However, we discovered that GPT-4V frequently presents flawed rationales in cases where it makes the correct final choices (35.5%), most prominent in image comprehension (27.2%). Regardless of GPT-4V's high accuracy in multi-choice questions, our findings emphasize the necessity for further in-depth evaluations of its rationales before integrating such multimodal AI models into clinical workflows. △ Less

Submitted 22 April, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Comments: Under review

Comprehensive analysis of synthetic learning applied to neonatal brain MRI segmentation

Authors: R Valabregue, F Girka, A Pron, F Rousseau, G Auzias

Abstract: Brain segmentation from neonatal MRI images is a very challenging task due to large changes in the shape of cerebral structures and variations in signal intensities reflecting the gestational process. In this context, there is a clear need for segmentation techniques that are robust to variations in image contrast and to the spatial configuration of anatomical structures. In this work, we evaluate… ▽ More Brain segmentation from neonatal MRI images is a very challenging task due to large changes in the shape of cerebral structures and variations in signal intensities reflecting the gestational process. In this context, there is a clear need for segmentation techniques that are robust to variations in image contrast and to the spatial configuration of anatomical structures. In this work, we evaluate the potential of synthetic learning, a contrast-independent model trained using synthetic images generated from the ground truth labels of very few subjects.We base our experiments on the dataset released by the developmental Human Connectome Project, for which high-quality T1- and T2-weighted images are available for more than 700 babies aged between 26 and 45 weeks post-conception. First, we confirm the impressive performance of a standard Unet trained on a few T2-weighted volumes, but also confirm that such models learn intensity-related features specific to the training domain. We then evaluate the synthetic learning approach and confirm its robustness to variations in image contrast by reporting the capacity of such a model to segment both T1- and T2-weighted images from the same individuals. However, we observe a clear influence of the age of the baby on the predictions. We improve the performance of this model by enriching the synthetic training set with realistic motion artifacts and over-segmentation of the white matter. Based on extensive visual assessment, we argue that the better performance of the model trained on real T2w data may be due to systematic errors in the ground truth. We propose an original experiment combining two definitions of the ground truth allowing us to show that learning from real data will reproduce any systematic bias from the training set, while synthetic models can avoid this limitation. Overall, our experiments confirm that synthetic learning is an effective solution for segmenting neonatal brain MRI. Our adapted synthetic learning approach combines key features that will be instrumental for large multi-site studies and clinical applications. △ Less

Submitted 11 September, 2023; originally announced September 2023.

Less Likely Brainstorming: Using Language Models to Generate Alternative Hypotheses

Authors: Liyan Tang, Yifan Peng, Yanshan Wang, Ying Ding, Greg Durrett, Justin F. Rousseau

Abstract: A human decision-maker benefits the most from an AI assistant that corrects for their biases. For problems such as generating interpretation of a radiology report given findings, a system predicting only highly likely outcomes may be less useful, where such outcomes are already obvious to the user. To alleviate biases in human decision-making, it is worth considering a broad differential diagnosis… ▽ More A human decision-maker benefits the most from an AI assistant that corrects for their biases. For problems such as generating interpretation of a radiology report given findings, a system predicting only highly likely outcomes may be less useful, where such outcomes are already obvious to the user. To alleviate biases in human decision-making, it is worth considering a broad differential diagnosis, going beyond the most likely options. We introduce a new task, "less likely brainstorming," that asks a model to generate outputs that humans think are relevant but less likely to happen. We explore the task in two settings: a brain MRI interpretation generation setting and an everyday commonsense reasoning setting. We found that a baseline approach of training with less likely hypotheses as targets generates outputs that humans evaluate as either likely or irrelevant nearly half of the time; standard MLE training is not effective. To tackle this problem, we propose a controlled text generation method that uses a novel contrastive learning strategy to encourage models to differentiate between generating likely and less likely outputs according to humans. We compare our method with several state-of-the-art controlled text generation models via automatic and human evaluations and show that our models' capability of generating less likely outputs is improved. △ Less

Submitted 30 May, 2023; originally announced May 2023.

Comments: Accepted to ACL (Findings) 2023

Analyzing Impact of Socio-Economic Factors on COVID-19 Mortality Prediction Using SHAP Value

Authors: Redoan Rahman, Jooyeong Kang, Justin F Rousseau, Ying Ding

Abstract: This paper applies multiple machine learning (ML) algorithms to a dataset of de-identified COVID-19 patients provided by the COVID-19 Research Database. The dataset consists of 20,878 COVID-positive patients, among which 9,177 patients died in the year 2020. This paper aims to understand and interpret the association of socio-economic characteristics of patients with their mortality instead of max… ▽ More This paper applies multiple machine learning (ML) algorithms to a dataset of de-identified COVID-19 patients provided by the COVID-19 Research Database. The dataset consists of 20,878 COVID-positive patients, among which 9,177 patients died in the year 2020. This paper aims to understand and interpret the association of socio-economic characteristics of patients with their mortality instead of maximizing prediction accuracy. According to our analysis, a patients households annual and disposable income, age, education, and employment status significantly impacts a machine learning models prediction. We also observe several individual patient data, which gives us insight into how the feature values impact the prediction for that data point. This paper analyzes the global and local interpretation of machine learning models on socio-economic data of COVID patients. △ Less

Submitted 27 February, 2023; originally announced March 2023.

Comments: 10 pages, 10 figures, American Medical Informatics Association(AMIA) Annual Conference 2022, Washington DC, USA, Nov 5-9, 2022

Journal ref: AMIA 2022 Annual Symposium

A Chisel Framework for Flexible Design Space Exploration through a Functional Approach

Authors: Bruno Ferres, Olivier Muller, Frédéric Rousseau

Abstract: As the need for efficient digital circuits is ever growing in the industry, the design of such systems remains daunting, requiring both expertise and time. In an attempt to close the gap between software development and hardware design, powerful features such as functional and object-oriented programming have been used to define new languages, known as Hardware Construction Languages. In this arti… ▽ More As the need for efficient digital circuits is ever growing in the industry, the design of such systems remains daunting, requiring both expertise and time. In an attempt to close the gap between software development and hardware design, powerful features such as functional and object-oriented programming have been used to define new languages, known as Hardware Construction Languages. In this article, we investigate the usage of such languages - more precisely, of Chisel - in the context of Design Space Exploration, and propose a novel design methodology to build custom and adaptable design flows. We apply a functional approach to define flexible strategies for design space exploration, based on combinations of basic exploration steps, and provide a proof-of-concept framework along with a library of basic strategies. We demonstrate our methodology through several use cases, illustrating how various metrics of interest can be considered to build exploration processes - in particular, we provide a quality of service-driven exploration example. The methodology presented in this work makes use of designers' expertise to reduce the time required for hardware design, in particular for Design Space Exploration, and its application should ease digital design and enhance hardware developpers' productivity. △ Less

Submitted 24 February, 2023; originally announced February 2023.

Using Explainable AI to Cross-Validate Socio-economic Disparities Among Covid-19 Patient Mortality

Authors: Li Shi, Redoan Rahman, Esther Melamed, Jacek Gwizdka, Justin F. Rousseau, Ying Ding

Abstract: This paper applies eXplainable Artificial Intelligence (XAI) methods to investigate the socioeconomic disparities in COVID patient mortality. An Extreme Gradient Boosting (XGBoost) prediction model is built based on a de-identified Austin area hospital dataset to predict the mortality of COVID-19 patients. We apply two XAI methods, Shapley Additive exPlanations (SHAP) and Locally Interpretable Mod… ▽ More This paper applies eXplainable Artificial Intelligence (XAI) methods to investigate the socioeconomic disparities in COVID patient mortality. An Extreme Gradient Boosting (XGBoost) prediction model is built based on a de-identified Austin area hospital dataset to predict the mortality of COVID-19 patients. We apply two XAI methods, Shapley Additive exPlanations (SHAP) and Locally Interpretable Model Agnostic Explanations (LIME), to compare the global and local interpretation of feature importance. This paper demonstrates the advantages of using XAI which shows the feature importance and decisive capability. Furthermore, we use the XAI methods to cross-validate their interpretations for individual patients. The XAI models reveal that Medicare financial class, older age, and gender have high impact on the mortality prediction. We find that LIME local interpretation does not show significant differences in feature importance comparing to SHAP, which suggests pattern confirmation. This paper demonstrates the importance of XAI methods in cross-validation of feature attributions. △ Less

Submitted 16 February, 2023; originally announced February 2023.

Comments: AMIA 2023 Informatics Summit, March 13-16, Seattle, WA, USA. 10 pages

Journal ref: AMIA 2023 Informatics Summit

Attend Who is Weak: Pruning-assisted Medical Image Localization under Sophisticated and Implicit Imbalances

Authors: Ajay Jaiswal, Tianlong Chen, Justin F. Rousseau, Yifan Peng, Ying Ding, Zhangyang Wang

Abstract: Deep neural networks (DNNs) have rapidly become a \textit{de facto} choice for medical image understanding tasks. However, DNNs are notoriously fragile to the class imbalance in image classification. We further point out that such imbalance fragility can be amplified when it comes to more sophisticated tasks such as pathology localization, as imbalances in such problems can have highly complex and… ▽ More Deep neural networks (DNNs) have rapidly become a \textit{de facto} choice for medical image understanding tasks. However, DNNs are notoriously fragile to the class imbalance in image classification. We further point out that such imbalance fragility can be amplified when it comes to more sophisticated tasks such as pathology localization, as imbalances in such problems can have highly complex and often implicit forms of presence. For example, different pathology can have different sizes or colors (w.r.t.the background), different underlying demographic distributions, and in general different difficulty levels to recognize, even in a meticulously curated balanced distribution of training data. In this paper, we propose to use pruning to automatically and adaptively identify \textit{hard-to-learn} (HTL) training samples, and improve pathology localization by attending them explicitly, during training in \textit{supervised, semi-supervised, and weakly-supervised} settings. Our main inspiration is drawn from the recent finding that deep classification models have difficult-to-memorize samples and those may be effectively exposed through network pruning \cite{hooker2019compressed} - and we extend such observation beyond classification for the first time. We also present an interesting demographic analysis which illustrates HTLs ability to capture complex demographic imbalances. Our extensive experiments on the Skin Lesion Localization task in multiple training settings by paying additional attention to HTLs show significant improvement of localization performance by $\sim$2-3\%. △ Less

Submitted 5 December, 2022; originally announced December 2022.

Comments: Accepted in WACV 2023

RoS-KD: A Robust Stochastic Knowledge Distillation Approach for Noisy Medical Imaging

Authors: Ajay Jaiswal, Kumar Ashutosh, Justin F Rousseau, Yifan Peng, Zhangyang Wang, Ying Ding

Abstract: AI-powered Medical Imaging has recently achieved enormous attention due to its ability to provide fast-paced healthcare diagnoses. However, it usually suffers from a lack of high-quality datasets due to high annotation cost, inter-observer variability, human annotator error, and errors in computer-generated labels. Deep learning models trained on noisy labelled datasets are sensitive to the noise… ▽ More AI-powered Medical Imaging has recently achieved enormous attention due to its ability to provide fast-paced healthcare diagnoses. However, it usually suffers from a lack of high-quality datasets due to high annotation cost, inter-observer variability, human annotator error, and errors in computer-generated labels. Deep learning models trained on noisy labelled datasets are sensitive to the noise type and lead to less generalization on the unseen samples. To address this challenge, we propose a Robust Stochastic Knowledge Distillation (RoS-KD) framework which mimics the notion of learning a topic from multiple sources to ensure deterrence in learning noisy information. More specifically, RoS-KD learns a smooth, well-informed, and robust student manifold by distilling knowledge from multiple teachers trained on overlap** subsets of training data. Our extensive experiments on popular medical imaging classification tasks (cardiopulmonary disease and lesion classification) using real-world datasets, show the performance benefit of RoS-KD, its ability to distill knowledge from many popular large networks (ResNet-50, DenseNet-121, MobileNet-V2) in a comparatively small network, and its robustness to adversarial attacks (PGD, FSGM). More specifically, RoS-KD achieves >2% and >4% improvement on F1-score for lesion classification and cardiopulmonary disease classification tasks, respectively, when the underlying student is ResNet-18 against recent competitive knowledge distillation baseline. Additionally, on cardiopulmonary disease classification task, RoS-KD outperforms most of the SOTA baselines by ~1% gain in AUC score. △ Less

Submitted 2 December, 2022; v1 submitted 15 October, 2022; originally announced October 2022.

Comments: Accepted in ICDM 2022

Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again

Authors: Ajay Jaiswal, Peihao Wang, Tianlong Chen, Justin F. Rousseau, Ying Ding, Zhangyang Wang

Abstract: Despite the enormous success of Graph Convolutional Networks (GCNs) in modeling graph-structured data, most of the current GCNs are shallow due to the notoriously challenging problems of over-smoothening and information squashing along with conventional difficulty caused by vanishing gradients and over-fitting. Previous works have been primarily focused on the study of over-smoothening and over-sq… ▽ More Despite the enormous success of Graph Convolutional Networks (GCNs) in modeling graph-structured data, most of the current GCNs are shallow due to the notoriously challenging problems of over-smoothening and information squashing along with conventional difficulty caused by vanishing gradients and over-fitting. Previous works have been primarily focused on the study of over-smoothening and over-squashing phenomena in training deep GCNs. Surprisingly, in comparison with CNNs/RNNs, very limited attention has been given to understanding how healthy gradient flow can benefit the trainability of deep GCNs. In this paper, firstly, we provide a new perspective of gradient flow to understand the substandard performance of deep GCNs and hypothesize that by facilitating healthy gradient flow, we can significantly improve their trainability, as well as achieve state-of-the-art (SOTA) level performance from vanilla-GCNs. Next, we argue that blindly adopting the Glorot initialization for GCNs is not optimal, and derive a topology-aware isometric initialization scheme for vanilla-GCNs based on the principles of isometry. Additionally, contrary to ad-hoc addition of skip-connections, we propose to use gradient-guided dynamic rewiring of vanilla-GCNs} with skip connections. Our dynamic rewiring method uses the gradient flow within each layer during training to introduce on-demand skip-connections adaptively. We provide extensive empirical evidence across multiple datasets that our methods improve gradient flow in deep vanilla-GCNs and significantly boost their performance to comfortably compete and outperform many fancy state-of-the-art methods. Codes are available at: https://github.com/VITA-Group/GradientGCN. △ Less

Submitted 14 October, 2022; originally announced October 2022.

Comments: Advances in Neural Information Processing Systems (NeurIPS), 2022

Turning Normalizing Flows into Monge Maps with Geodesic Gaussian Preserving Flows

Authors: Guillaume Morel, Lucas Drumetz, Simon Benaïchouche, Nicolas Courty, François Rousseau

Abstract: Normalizing Flows (NF) are powerful likelihood-based generative models that are able to trade off between expressivity and tractability to model complex densities. A now well established research avenue leverages optimal transport (OT) and looks for Monge maps, i.e. models with minimal effort between the source and target distributions. This paper introduces a method based on Brenier's polar facto… ▽ More Normalizing Flows (NF) are powerful likelihood-based generative models that are able to trade off between expressivity and tractability to model complex densities. A now well established research avenue leverages optimal transport (OT) and looks for Monge maps, i.e. models with minimal effort between the source and target distributions. This paper introduces a method based on Brenier's polar factorization theorem to transform any trained NF into a more OT-efficient version without changing the final density. We do so by learning a rearrangement of the source (Gaussian) distribution that minimizes the OT cost between the source and the final density. We further constrain the path leading to the estimated Monge map to lie on a geodesic in the space of volume-preserving diffeomorphisms thanks to Euler's equations. The proposed method leads to smooth flows with reduced OT cost for several existing models without affecting the model performance. △ Less

Submitted 14 April, 2023; v1 submitted 22 September, 2022; originally announced September 2022.

Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors

Authors: Liyan Tang, Tanya Goyal, Alexander R. Fabbri, Philippe Laban, Jiacheng Xu, Semih Yavuz, Wojciech Kryściński, Justin F. Rousseau, Greg Durrett

Abstract: The propensity of abstractive summarization models to make factual errors has been studied extensively, including design of metrics to detect factual errors and annotation of errors in current systems' outputs. However, the ever-evolving nature of summarization systems, metrics, and annotated benchmarks makes factuality evaluation a moving target, and drawing clear comparisons among metrics has be… ▽ More The propensity of abstractive summarization models to make factual errors has been studied extensively, including design of metrics to detect factual errors and annotation of errors in current systems' outputs. However, the ever-evolving nature of summarization systems, metrics, and annotated benchmarks makes factuality evaluation a moving target, and drawing clear comparisons among metrics has become increasingly difficult. In this work, we aggregate factuality error annotations from nine existing datasets and stratify them according to the underlying summarization model. We compare performance of state-of-the-art factuality metrics, including recent ChatGPT-based metrics, on this stratified benchmark and show that their performance varies significantly across different types of summarization models. Critically, our analysis shows that much of the recent improvement in the factuality detection space has been on summaries from older (pre-Transformer) models instead of more relevant recent summarization models. We further perform a finer-grained analysis per error-type and find similar performance variance across error types for different factuality metrics. Our results show that no one metric is superior in all settings or for all error types, and we provide recommendations for best practices given these insights. △ Less

Submitted 25 May, 2023; v1 submitted 25 May, 2022; originally announced May 2022.

Comments: Accepted to ACL 2023

SCALP -- Supervised Contrastive Learning for Cardiopulmonary Disease Classification and Localization in Chest X-rays using Patient Metadata

Authors: Ajay Jaiswal, Tianhao Li, Cyprian Zander, Yan Han, Justin F. Rousseau, Yifan Peng, Ying Ding

Abstract: Computer-aided diagnosis plays a salient role in more accessible and accurate cardiopulmonary diseases classification and localization on chest radiography. Millions of people get affected and die due to these diseases without an accurate and timely diagnosis. Recently proposed contrastive learning heavily relies on data augmentation, especially positive data augmentation. However, generating clin… ▽ More Computer-aided diagnosis plays a salient role in more accessible and accurate cardiopulmonary diseases classification and localization on chest radiography. Millions of people get affected and die due to these diseases without an accurate and timely diagnosis. Recently proposed contrastive learning heavily relies on data augmentation, especially positive data augmentation. However, generating clinically-accurate data augmentations for medical images is extremely difficult because the common data augmentation methods in computer vision, such as sharp, blur, and crop operations, can severely alter the clinical settings of medical images. In this paper, we proposed a novel and simple data augmentation method based on patient metadata and supervised knowledge to create clinically accurate positive and negative augmentations for chest X-rays. We introduce an end-to-end framework, SCALP, which extends the self-supervised contrastive approach to a supervised setting. Specifically, SCALP pulls together chest X-rays from the same patient (positive keys) and pushes apart chest X-rays from different patients (negative keys). In addition, it uses ResNet-50 along with the triplet-attention mechanism to identify cardiopulmonary diseases, and Grad-CAM++ to highlight the abnormal regions. Our extensive experiments demonstrate that SCALP outperforms existing baselines with significant margins in both classification and localization tasks. Specifically, the average classification AUCs improve from 82.8% (SOTA using DenseNet-121) to 83.9% (SCALP using ResNet-50), while the localization results improve on average by 3.7% over different IoU thresholds. △ Less

Submitted 27 October, 2021; originally announced October 2021.

Possible Contexts of Use for In Silico trials methodologies: a consensus-based review

Authors: Marco Viceconti, Luca Emili, Payman Afshari, Eulalie Courcelles, Cristina Curreli, Nele Famaey, Liesbet Geris, Marc Horner, Maria Cristina Jori, Alexander Kulesza, Axel Loewe, Michael Neidlin, Markus Reiterer, Cecile F. Rousseau, Giulia Russo, Simon J. Sonntag, Emmanuelle M. Voisin, Francesco Pappalardo

Abstract: The term "In Silico Trial" indicates the use of computer modelling and simulation to evaluate the safety and efficacy of a medical product, whether a drug, a medical device, a diagnostic product or an advanced therapy medicinal product. Predictive models are positioned as new methodologies for the development and the regulatory evaluation of medical products. New methodologies are qualified by reg… ▽ More The term "In Silico Trial" indicates the use of computer modelling and simulation to evaluate the safety and efficacy of a medical product, whether a drug, a medical device, a diagnostic product or an advanced therapy medicinal product. Predictive models are positioned as new methodologies for the development and the regulatory evaluation of medical products. New methodologies are qualified by regulators such as FDA and EMA through formal processes, where a first step is the definition of the Context of Use (CoU), which is a concise description of how the new methodology is intended to be used in the development and regulatory assessment process. As In Silico Trials are a disruptively innovative class of new methodologies, it is important to have a list of possible CoUs highlighting potential applications for the development of the relative regulatory science. This review paper presents the result of a consensus process that took place in the InSilicoWorld Community of Practice, an online forum for experts in in silico medicine. The experts involved identified 46 descriptions of possible CoUs which were organised into a candidate taxonomy of nine CoU categories. Examples of 31 CoUs were identified in the available literature; the remaining 15 should, for now, be considered speculative. △ Less

Submitted 28 March, 2021; originally announced March 2021.

Comments: Submitted to IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS

Gradients of Connectivity as Graph Fourier Bases of Brain Activity

Authors: Giulia Lioi, Vincent Gripon, Abdelbasset Brahim, François Rousseau, Nicolas Farrugia

Abstract: The application of graph theory to model the complex structure and function of the brain has shed new light on its organization and function, prompting the emergence of network neuroscience. Despite the tremendous progress that has been achieved in this field, still relatively few methods exploit the topology of brain networks to analyze brain activity. Recent attempts in this direction have lever… ▽ More The application of graph theory to model the complex structure and function of the brain has shed new light on its organization and function, prompting the emergence of network neuroscience. Despite the tremendous progress that has been achieved in this field, still relatively few methods exploit the topology of brain networks to analyze brain activity. Recent attempts in this direction have leveraged on graph spectral analysis and graph signal processing to decompose brain activity in connectivity eigenmodes or gradients. If results are promising in terms of interpretability and functional relevance, methodologies and terminology are sometimes confusing. The goals of this paper are twofold. First, we summarize recent contributions related to connectivity gradients and graph signal processing, and attempt a clarification of the terminology and methods used in the field, while pointing out current methodological limitations. Second, we discuss the perspective that the functional relevance of connectivity gradients could be fruitfully exploited by considering them as graph Fourier bases of brain activity. △ Less

Submitted 26 September, 2020; originally announced September 2020.

Comments: 14 pages, 1 figure, 2 Tables, submitted to Network Neuroscience

Learning Variational Data Assimilation Models and Solvers

Authors: Ronan Fablet, Bertrand Chapron, Lucas. Drumetz, Etienne Memin, Olivier Pannekoucke, Francois Rousseau

Abstract: This paper addresses variational data assimilation from a learning point of view. Data assimilation aims to reconstruct the time evolution of some state given a series of observations, possibly noisy and irregularly-sampled. Using automatic differentiation tools embedded in deep learning frameworks, we introduce end-to-end neural network architectures for data assimilation. It comprises two key co… ▽ More This paper addresses variational data assimilation from a learning point of view. Data assimilation aims to reconstruct the time evolution of some state given a series of observations, possibly noisy and irregularly-sampled. Using automatic differentiation tools embedded in deep learning frameworks, we introduce end-to-end neural network architectures for data assimilation. It comprises two key components: a variational model and a gradient-based solver both implemented as neural networks. A key feature of the proposed end-to-end learning architecture is that we may train the NN models using both supervised and unsupervised strategies. Our numerical experiments on Lorenz-63 and Lorenz-96 systems report significant gain w.r.t. a classic gradient-based minimization of the variational cost both in terms of reconstruction performance and optimization complexity. Intriguingly, we also show that the variational models issued from the true Lorenz-63 and Lorenz-96 ODE representations may not lead to the best reconstruction performance. We believe these results may open new research avenues for the specification of assimilation models in geoscience. △ Less

Submitted 25 July, 2020; originally announced July 2020.

Joint learning of variational representations and solvers for inverse problems with partially-observed data

Authors: Ronan Fablet, Lucas Drumetz, Francois Rousseau

Abstract: Designing appropriate variational regularization schemes is a crucial part of solving inverse problems, making them better-posed and guaranteeing that the solution of the associated optimization problem satisfies desirable properties. Recently, learning-based strategies have appeared to be very efficient for solving inverse problems, by learning direct inversion schemes or plug-and-play regularize… ▽ More Designing appropriate variational regularization schemes is a crucial part of solving inverse problems, making them better-posed and guaranteeing that the solution of the associated optimization problem satisfies desirable properties. Recently, learning-based strategies have appeared to be very efficient for solving inverse problems, by learning direct inversion schemes or plug-and-play regularizers from available pairs of true states and observations. In this paper, we go a step further and design an end-to-end framework allowing to learn actual variational frameworks for inverse problems in such a supervised setting. The variational cost and the gradient-based solver are both stated as neural networks using automatic differentiation for the latter. We can jointly learn both components to minimize the data reconstruction error on the true states. This leads to a data-driven discovery of variational models. We consider an application to inverse problems with incomplete datasets (image inpainting and multivariate time series interpolation). We experimentally illustrate that this framework can lead to a significant gain in terms of reconstruction performance, including w.r.t. the direct minimization of the variational formulation derived from the known generative model. △ Less

Submitted 5 June, 2020; originally announced June 2020.

Comments: 10 pages, 2 figures

Building a PubMed knowledge graph

Authors: Jian Xu, Sunkyu Kim, Min Song, Minbyul Jeong, Donghyeon Kim, Jaewoo Kang, Justin F. Rousseau, Xin Li, Weijia Xu, Vetle I. Torvik, Yi Bu, Chongyan Chen, Islam Akef Ebeid, Daifeng Li, Ying Ding

Abstract: PubMed is an essential resource for the medical domain, but useful concepts are either difficult to extract or are ambiguated, which has significantly hindered knowledge discovery. To address this issue, we constructed a PubMed knowledge graph (PKG) by extracting bio-entities from 29 million PubMed abstracts, disambiguating author names, integrating funding data through the National Institutes of… ▽ More PubMed is an essential resource for the medical domain, but useful concepts are either difficult to extract or are ambiguated, which has significantly hindered knowledge discovery. To address this issue, we constructed a PubMed knowledge graph (PKG) by extracting bio-entities from 29 million PubMed abstracts, disambiguating author names, integrating funding data through the National Institutes of Health (NIH) ExPORTER, collecting affiliation history and educational background of authors from ORCID, and identifying fine-grained affiliation data from MapAffil. Through the integration of the credible multi-source data, we could create connections among the bio-entities, authors, articles, affiliations, and funding. Data validation revealed that the BioBERT deep learning method of bio-entity extraction significantly outperformed the state-of-the-art models based on the F1 score (by 0.51%), with the author name disambiguation (AND) achieving a F1 score of 98.09%. PKG can trigger broader innovations, not only enabling us to measure scholarly impact, knowledge usage, and knowledge transfer, but also assisting us in profiling authors and organizations based on their connections with bio-entities. The PKG is freely available on Figshare (https://figshare.com/s/6327a55355fc2c99f3a2, simplified version that exclude PubMed raw data) and TACC website (http://er.tacc.utexas.edu/datasets/ped, full version). △ Less

Submitted 15 May, 2020; v1 submitted 8 May, 2020; originally announced May 2020.

Comments: 19 pages, 5 figures, 14 tables

A fast and memory-efficient algorithm for smooth interpolation of polyrigid transformations: application to human joint tracking

Authors: K. Makki, B. Borotikar, M. Garetier, S. Brochard, D. Ben Salem, F. Rousseau

Abstract: The log Euclidean polyrigid registration framework provides a way to smoothly estimate and interpolate poly-rigid/affine transformations for which the invertibility is guaranteed. This powerful and flexible mathematical framework is currently being used to track the human joint dynamics by first imposing bone rigidity constraints in order to synthetize the spatio-temporal joint deformations later.… ▽ More The log Euclidean polyrigid registration framework provides a way to smoothly estimate and interpolate poly-rigid/affine transformations for which the invertibility is guaranteed. This powerful and flexible mathematical framework is currently being used to track the human joint dynamics by first imposing bone rigidity constraints in order to synthetize the spatio-temporal joint deformations later. However, since no closed-form exists, then a computationally expensive integration of ordinary differential equations (ODEs) is required to perform image registration using this framework. To tackle this problem, the exponential map for solving these ODEs is computed using the scaling and squaring method in the literature. In this paper, we propose an algorithm using a matrix diagonalization based method for smooth interpolation of homogeneous polyrigid transformations of human joints during motion. The use of this alternative computational approach to integrate ODEs is well motivated by the fact that bone rigid transformations satisfy the mechanical constraints of human joint motion, which provide conditions that guarantee the diagonalizability of local bone transformations and consequently of the resulting joint transformations. In a comparison with the scaling and squaring method, we discuss the usefulness of the matrix eigendecomposition technique which reduces significantly the computational burden associated with the computation of matrix exponential over a dense regular grid. Finally, we have applied the method to enhance the temporal resolution of dynamic MRI sequences of the ankle joint. To conclude, numerical experiments show that the eigendecomposition method is more capable of balancing the trade-off between accuracy, computation time, and memory requirements. △ Less

Submitted 8 June, 2020; v1 submitted 28 April, 2020; originally announced May 2020.

Abdominal multi-organ segmentation with cascaded convolutional and adversarial deep networks

Authors: Pierre-Henri Conze, Ali Emre Kavur, Emilie Cornec-Le Gall, Naciye Sinem Gezer, Yannick Le Meur, M. Alper Selver, François Rousseau

Abstract: Objective : Abdominal anatomy segmentation is crucial for numerous applications from computer-assisted diagnosis to image-guided surgery. In this context, we address fully-automated multi-organ segmentation from abdominal CT and MR images using deep learning. Methods: The proposed model extends standard conditional generative adversarial networks. Additionally to the discriminator which enforces t… ▽ More Objective : Abdominal anatomy segmentation is crucial for numerous applications from computer-assisted diagnosis to image-guided surgery. In this context, we address fully-automated multi-organ segmentation from abdominal CT and MR images using deep learning. Methods: The proposed model extends standard conditional generative adversarial networks. Additionally to the discriminator which enforces the model to create realistic organ delineations, it embeds cascaded partially pre-trained convolutional encoder-decoders as generator. Encoder fine-tuning from a large amount of non-medical images alleviates data scarcity limitations. The network is trained end-to-end to benefit from simultaneous multi-level segmentation refinements using auto-context. Results : Employed for healthy liver, kidneys and spleen segmentation, our pipeline provides promising results by outperforming state-of-the-art encoder-decoder schemes. Followed for the Combined Healthy Abdominal Organ Segmentation (CHAOS) challenge organized in conjunction with the IEEE International Symposium on Biomedical Imaging 2019, it gave us the first rank for three competition categories: liver CT, liver MR and multi-organ MR segmentation. Conclusion : Combining cascaded convolutional and adversarial networks strengthens the ability of deep learning pipelines to automatically delineate multiple abdominal organs, with good generalization capability. Significance : The comprehensive evaluation provided suggests that better guidance could be achieved to help clinicians in abdominal image interpretation and clinical decision making. △ Less

Submitted 26 January, 2020; originally announced January 2020.

End-to-end learning of energy-based representations for irregularly-sampled signals and images

Authors: Ronan Fablet, Lucas Drumetz, François Rousseau

Abstract: For numerous domains, including for instance earth observation, medical imaging, astrophysics,..., available image and signal datasets often involve irregular space-time sampling patterns and large missing data rates. These sampling properties may be critical to apply state-of-the-art learning-based (e.g., auto-encoders, CNNs,...), fully benefit from the available large-scale observations and reac… ▽ More For numerous domains, including for instance earth observation, medical imaging, astrophysics,..., available image and signal datasets often involve irregular space-time sampling patterns and large missing data rates. These sampling properties may be critical to apply state-of-the-art learning-based (e.g., auto-encoders, CNNs,...), fully benefit from the available large-scale observations and reach breakthroughs in the reconstruction and identification of processes of interest. In this paper, we address the end-to-end learning of representations of signals, images and image sequences from irregularly-sampled data, i.e. when the training data involved missing data. From an analogy to Bayesian formulation, we consider energy-based representations. Two energy forms are investigated: one derived from auto-encoders and one relating to Gibbs priors. The learning stage of these energy-based representations (or priors) involve a joint interpolation issue, which amounts to solving an energy minimization problem under observation constraints. Using a neural-network-based implementation of the considered energy forms, we can state an end-to-end learning scheme from irregularly-sampled data. We demonstrate the relevance of the proposed representations for different case-studies: namely, multivariate time series, 2D images and image sequences. △ Less

Submitted 1 October, 2019; originally announced October 2019.

On early brain folding patterns using biomechanical growth modeling

Authors: Xiaoyu Wang, Amine Bohi, Mariam Al Harrach, Mickael Dinomais, Julien Lefèvre, François Rousseau

Abstract: Abnormal cortical folding patterns may be related to neurodevelopmental disorders such as lissencephaly and polymicrogyria. In this context, computational modeling is a powerful tool to provide a better understanding of the early brain folding process. Recent studies based on biomechanical modeling have shown that mechanical forces play a crucial role in the formation of cortical convolutions. How… ▽ More Abnormal cortical folding patterns may be related to neurodevelopmental disorders such as lissencephaly and polymicrogyria. In this context, computational modeling is a powerful tool to provide a better understanding of the early brain folding process. Recent studies based on biomechanical modeling have shown that mechanical forces play a crucial role in the formation of cortical convolutions. However, the correlation between simulation results and biological facts, and the effect of physical parameters in these models remain unclear. In this paper, we propose a new brain longitudinal length growth model to improve brain model growth. In addition, we investigate the effect of the initial cortical thickness on folding patterns, quantifying the folds by the surface-based three-dimensional gyrification index and a spectral analysis of gyrification. The results tend to show that the use of such biomechanical models could highlight the links between neurodevelopmental diseases and physical parameters. △ Less

Submitted 7 May, 2019; originally announced May 2019.

Comments: 4 pages, 4 figures, 41st EMB Conference

Global Perturbation of Initial Geometry in a Biomechanical Model of Cortical Morphogenesis

Authors: Amine Bohi, Xiaoyu Wang, Mariam Al Harrach, Mickael Dinomais, François Rousseau, Julien Lefèvre

Abstract: Cortical folding pattern is a main characteristic of the geometry of the human brain which is formed by gyri (ridges) and sulci (grooves). Several biological hypotheses have suggested different mechanisms that attempt to explain the development of cortical folding and its abnormal evolutions. Based on these hypotheses, biomechanical models of cortical folding have been proposed. In this work, we c… ▽ More Cortical folding pattern is a main characteristic of the geometry of the human brain which is formed by gyri (ridges) and sulci (grooves). Several biological hypotheses have suggested different mechanisms that attempt to explain the development of cortical folding and its abnormal evolutions. Based on these hypotheses, biomechanical models of cortical folding have been proposed. In this work, we compare biomechanical simulations for several initial conditions by using an adaptive spherical parameterization approach. Our approach allows us to study and explore one of the most potential sources of reproducible cortical folding pattern: the specification of initial geometry of the brain. △ Less

Submitted 7 May, 2019; v1 submitted 11 February, 2019; originally announced February 2019.

Comments: 4 pages 2 columns (IEEE style), 41st EMB Conference

Residual Networks as Geodesic Flows of Diffeomorphisms

Authors: Francois Rousseau, Ronan Fablet

Abstract: This paper addresses the understanding and characterization of residual networks (ResNet), which are among the state-of-the-art deep learning architectures for a variety of supervised learning problems. We focus on the map** component of ResNets, which map the embedding space towards a new unknown space where the prediction or classification can be stated according to linear criteria. We show th… ▽ More This paper addresses the understanding and characterization of residual networks (ResNet), which are among the state-of-the-art deep learning architectures for a variety of supervised learning problems. We focus on the map** component of ResNets, which map the embedding space towards a new unknown space where the prediction or classification can be stated according to linear criteria. We show that this map** component can be regarded as the numerical implementation of continuous flows of diffeomorphisms governed by ordinary differential equations. Especially, ResNets with shared weights are fully characterized as numerical approximation of exponential diffeomorphic operators. We stress both theoretically and numerically the relevance of the enforcement of diffeormorphic properties and the importance of numerical issues to make consistent the continuous formulation and the discretized ResNet implementation. We further discuss the resulting theoretical and computational insights on ResNet architectures. △ Less

Submitted 22 June, 2018; v1 submitted 24 May, 2018; originally announced May 2018.

Demonstration of a context-switch method for heterogeneous reconfigurable systems

Authors: Arief Wicaksana, Alban Bourge, Olivier Muller, Frédéric Rousseau

Abstract: Nowadays, FPGAs are integrated in high-performance computing systems, servers, or even used as accelerators in System-on-Chip (SoC) platforms. Since the execution is performed in hardware, FPGA gives much higher performance and lower energy consumption compared to most microprocessor-based systems. However, the room to improve FPGA performance still exists, e.g. when it is used by multiple users.… ▽ More Nowadays, FPGAs are integrated in high-performance computing systems, servers, or even used as accelerators in System-on-Chip (SoC) platforms. Since the execution is performed in hardware, FPGA gives much higher performance and lower energy consumption compared to most microprocessor-based systems. However, the room to improve FPGA performance still exists, e.g. when it is used by multiple users. In multi-user approaches, FPGA resources are shared between several users. Therefore, one must be able to interrupt a running circuit at any given time and continue the task at will. An image of the state of the running circuit (context) is saved during interruption and restored when the execution is continued. The ability to extract and restore the context is known as context-switch.In the previous work [1], an automatic checkpoint selection method is proposed for circuit generation targeting reconfigurable systems. The method relies on static analysis of the finite state machine of a circuit to select the checkpoint states. States with minimum overhead will be selected as checkpoints, which allow optimal context save and restore. The maximum time to reach a checkpoint will be defined by the user and consideredas the context-switch latency. The method is implemented in C code and integrated as plugin in a free and open-source High-Level Synthesis tool AUGH [2]. △ Less

Submitted 18 November, 2016; originally announced November 2016.

Journal ref: 2016 26th International Conference on Field Programmable Logic and Applications (FPL), Aug 2016, Lausanne, Switzerland. pp.1 - 1, 2016

DTLS Performance in Duty-Cycled Networks

Authors: Malisa Vucinic, Bernard Tourancheau, Thomas Watteyne, Franck Rousseau, Andrzej Duda, Roberto Guizzetti, Laurent Damon

Abstract: The Datagram Transport Layer Security (DTLS) protocol is the IETF standard for securing the Internet of Things. The Constrained Application Protocol, ZigBee IP, and Lightweight Machine-to-Machine (LWM2M) mandate its use for securing application traffic. There has been much debate in both the standardization and research communities on the applicability of DTLS to constrained environments. The main… ▽ More The Datagram Transport Layer Security (DTLS) protocol is the IETF standard for securing the Internet of Things. The Constrained Application Protocol, ZigBee IP, and Lightweight Machine-to-Machine (LWM2M) mandate its use for securing application traffic. There has been much debate in both the standardization and research communities on the applicability of DTLS to constrained environments. The main concerns are the communication overhead and latency of the DTLS handshake, and the memory footprint of a DTLS implementation. This paper provides a thorough performance evaluation of DTLS in different duty-cycled networks through real-world experimentation, emulation and analysis. In particular, we measure the duration of the DTLS handshake when using three duty cycling link-layer protocols: preamble-sampling, the IEEE 802.15.4 beacon-enabled mode and the IEEE 802.15.4e Time Slotted Channel Hop** mode. The reported results demonstrate surprisingly poor performance of DTLS in radio duty-cycled networks. Because a DTLS client and a server exchange more than 10 signaling packets, the DTLS handshake takes between a handful of seconds and several tens of seconds, with similar results for different duty cycling protocols. Moreover, because of their limited memory, typical constrained nodes can only maintain 3-5 simultaneous DTLS sessions, which highlights the need for using DTLS parsimoniously. △ Less

Submitted 21 July, 2015; originally announced July 2015.

Comments: International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC - 2015), IEEE, IEEE, 2015, http://pimrc2015.eee.hku.hk/index.html

Topology Construction in RPL Networks over Beacon-Enabled 802.15.4

Authors: Malisa Vucinic, Gabriele Romaniello, Laurene Guelorget, Bernard Tourancheau, Franck Rousseau, Olivier Alphand, Andrzej Duda, Laurent Damon

Abstract: In this paper, we propose a new scheme that allows coupling beacon-enabled IEEE 802.15.4 with the RPL routing protocol while kee** full compliance with both standards. We provide a means for RPL to pass the routing information to Layer 2 before the 802.15.4 topology is created by encapsulating RPL DIO messages in beacon frames. The scheme takes advantage of 802.15.4 command frames to solicit RPL… ▽ More In this paper, we propose a new scheme that allows coupling beacon-enabled IEEE 802.15.4 with the RPL routing protocol while kee** full compliance with both standards. We provide a means for RPL to pass the routing information to Layer 2 before the 802.15.4 topology is created by encapsulating RPL DIO messages in beacon frames. The scheme takes advantage of 802.15.4 command frames to solicit RPL DIO messages. The effect of the command frames is to reset the Trickle timer that governs sending DIO messages. We provide a detailed analysis of the overhead incurred by the proposed scheme to understand topology construction costs. We have evaluated the scheme using Contiki and the instruction-level Cooja simulator and compared our results against the most common scheme used for dissemination of the upper-layer information in beacon-enabled PANs. The results show energy savings during the topology construction phase and in the steady state. △ Less

Submitted 30 April, 2014; originally announced April 2014.

Journal ref: 19th IEEE Symposium on Computers and Communications (IEEE ISCC 2014) (2014)

OSCAR: Object Security Architecture for the Internet of Things

Authors: Malisa Vucinic, Bernard Tourancheau, Franck Rousseau, Andrzej Duda, Laurent Damon, Roberto Guizzetti

Abstract: Billions of smart, but constrained objects wirelessly connected to the global network require novel paradigms in network design. New protocol standards, tailored to constrained devices, have been designed taking into account requirements such as asynchronous application traffic, need for caching, and group communication. The existing connection oriented security architecture is not able to keep up… ▽ More Billions of smart, but constrained objects wirelessly connected to the global network require novel paradigms in network design. New protocol standards, tailored to constrained devices, have been designed taking into account requirements such as asynchronous application traffic, need for caching, and group communication. The existing connection oriented security architecture is not able to keep up---first, in terms of the supported features, but also in terms of the scale and resulting latency on small constrained devices. In this paper, we propose an architecture that leverages the security concepts both from content-centric and traditional connection-oriented approaches. We rely on secure channels established by means of (D)TLS for key exchange, but we get rid of the notion of the 'state' among communicating entities. We provide a mechanism to protect from replay attacks by coupling our scheme with the CoAP application protocol. Our object-based security architecture (OSCAR) intrinsically supports caching and multicast, and does not affect the radio duty-cycling operation of constrained objects. We evaluate OSCAR in two cases: 802.15.4 Low Power and Lossy Networks (LLN) and Machine-to-Machine (M2M) communication for two different hardware platforms and MAC layers on a real testbed and using the Cooja emulator. We show significant energy savings at constrained servers and reasonable delays. We also discuss the applicability of OSCAR to Smart City deployments. △ Less

Submitted 30 April, 2014; originally announced April 2014.

Journal ref: 15th IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks (IEEE WoWMoM 2014) (2014)

Using temporal IDF for efficient novelty detection in text streams

Authors: Margarita Karkali, Francois Rousseau, Alexandros Ntoulas, Michalis Vazirgiannis

Abstract: Novelty detection in text streams is a challenging task that emerges in quite a few different scenarios, ranging from email thread filtering to RSS news feed recommendation on a smartphone. An efficient novelty detection algorithm can save the user a great deal of time and resources when browsing through relevant yet usually previously-seen content. Most of the recent research on detection of nove… ▽ More Novelty detection in text streams is a challenging task that emerges in quite a few different scenarios, ranging from email thread filtering to RSS news feed recommendation on a smartphone. An efficient novelty detection algorithm can save the user a great deal of time and resources when browsing through relevant yet usually previously-seen content. Most of the recent research on detection of novel documents in text streams has been building upon either geometric distances or distributional similarities, with the former typically performing better but being much slower due to the need of comparing an incoming document with all the previously-seen ones. In this paper, we propose a new approach to novelty detection in text streams. We describe a resource-aware mechanism that is able to handle massive text streams such as the ones present today thanks to the burst of social media and the emergence of the Web as the main source of information. We capitalize on the historical Inverse Document Frequency (IDF) that was known for capturing well term specificity and we show that it can be used successfully at the document level as a measure of document novelty. This enables us to avoid similarity comparisons with previous documents in the text stream, thus scaling better and leading to faster execution times. Moreover, as the collection of documents evolves over time, we use a temporal variant of IDF not only to maintain an efficient representation of what has already been seen but also to decay the document frequencies as the time goes by. We evaluate the performance of the proposed approach on a real-world news articles dataset created for this task. The results show that the proposed method outperforms all of the baselines while managing to operate efficiently in terms of time complexity and memory usage, which are of great importance in a mobile setting scenario. △ Less

Submitted 9 November, 2014; v1 submitted 7 January, 2014; originally announced January 2014.

Comments: 30 pages

EURETILE 2010-2012 summary: first three years of activity of the European Reference Tiled Experiment

Authors: Pier Stanislao Paolucci, Iuliana Bacivarov, Gert Goossens, Rainer Leupers, Frédéric Rousseau, Christoph Schumacher, Lothar Thiele, Piero Vicini

Abstract: This is the summary of first three years of activity of the EURETILE FP7 project 247846. EURETILE investigates and implements brain-inspired and fault-tolerant foundational innovations to the system architecture of massively parallel tiled computer architectures and the corresponding programming paradigm. The execution targets are a many-tile HW platform, and a many-tile simulator. A set of SW pro… ▽ More This is the summary of first three years of activity of the EURETILE FP7 project 247846. EURETILE investigates and implements brain-inspired and fault-tolerant foundational innovations to the system architecture of massively parallel tiled computer architectures and the corresponding programming paradigm. The execution targets are a many-tile HW platform, and a many-tile simulator. A set of SW process - HW tile map** candidates is generated by the holistic SW tool-chain using a combination of analytic and bio-inspired methods. The Hardware dependent Software is then generated, providing OS services with maximum efficiency/minimal overhead. The many-tile simulator collects profiling data, closing the loop of the SW tool chain. Fine-grain parallelism inside processes is exploited by optimized intra-tile compilation techniques, but the project focus is above the level of the elementary tile. The elementary HW tile is a multi-processor, which includes a fault tolerant Distributed Network Processor (for inter-tile communication) and ASIP accelerators. Furthermore, EURETILE investigates and implements the innovations for equip** the elementary HW tile with high-bandwidth, low-latency brain-like inter-tile communication emulating 3 levels of connection hierarchy, namely neural columns, cortical areas and cortex, and develops a dedicated cortical simulation benchmark: DPSNN-STDP (Distributed Polychronous Spiking Neural Net with synaptic Spiking Time Dependent Plasticity). EURETILE leverages on the multi-tile HW paradigm and SW tool-chain developed by the FET-ACA SHAPES Integrated Project (2006-2009). △ Less

Submitted 7 May, 2013; originally announced May 2013.

Comments: 56 pages

ACM Class: C.1.4; C.3; B.7.2; F.2.2