-
Industrial Language-Image Dataset (ILID): Adapting Vision Foundation Models for Industrial Settings
Authors:
Keno Moenck,
Duc Trung Thieu,
Julian Koch,
Thorsten Schüppstuhl
Abstract:
In recent years, the upstream of Large Language Models (LLM) has also encouraged the computer vision community to work on substantial multimodal datasets and train models on a scale in a self-/semi-supervised manner, resulting in Vision Foundation Models (VFM), as, e.g., Contrastive Language-Image Pre-training (CLIP). The models generalize well and perform outstandingly on everyday objects or scen…
▽ More
In recent years, the upstream of Large Language Models (LLM) has also encouraged the computer vision community to work on substantial multimodal datasets and train models on a scale in a self-/semi-supervised manner, resulting in Vision Foundation Models (VFM), as, e.g., Contrastive Language-Image Pre-training (CLIP). The models generalize well and perform outstandingly on everyday objects or scenes, even on downstream tasks, tasks the model has not been trained on, while the application in specialized domains, as in an industrial context, is still an open research question. Here, fine-tuning the models or transfer learning on domain-specific data is unavoidable when objecting to adequate performance. In this work, we, on the one hand, introduce a pipeline to generate the Industrial Language-Image Dataset (ILID) based on web-crawled data; on the other hand, we demonstrate effective self-supervised transfer learning and discussing downstream tasks after training on the cheaply acquired ILID, which does not necessitate human labeling or intervention. With the proposed approach, we contribute by transferring approaches from state-of-the-art research around foundation models, transfer learning strategies, and applications to the industrial domain.
△ Less
Submitted 13 June, 2024;
originally announced June 2024.
-
Low-resource classification of mobility functioning information in clinical sentences using large language models
Authors:
Tuan Dung Le,
Thanh Duong,
Thanh Thieu
Abstract:
Objective: Function is increasingly recognized as an important indicator of whole-person health. This study evaluates the ability of publicly available large language models (LLMs) to accurately identify the presence of functioning information from clinical notes. We explore various strategies to improve the performance on this task. Materials and Methods: We collect a balanced binary classificati…
▽ More
Objective: Function is increasingly recognized as an important indicator of whole-person health. This study evaluates the ability of publicly available large language models (LLMs) to accurately identify the presence of functioning information from clinical notes. We explore various strategies to improve the performance on this task. Materials and Methods: We collect a balanced binary classification dataset of 1000 sentences from the Mobility NER dataset, which was curated from n2c2 clinical notes. For evaluation, we construct zero-shot and few-shot prompts to query the LLMs whether a given sentence contains mobility functioning information. Two sampling techniques, random sampling and k-nearest neighbor (kNN)-based sampling, are used to select the few-shot examples. Furthermore, we apply a parameter-efficient prompt-based fine-tuning method to the LLMs and evaluate their performance under various training settings. Results: Flan-T5-xxl outperforms all other models in both zero-shot and few-shot settings, achieving a F1 score of 0.865 with a single demonstrative example selected by kNN sampling. In prompt-based fine-tuning experiments, this foundation model also demonstrates superior performance across all low-resource settings, particularly achieving an impressive F1 score of 0.922 using the full training dataset. The smaller model, Flan-T5-xl, requires fine-tuning with only 2.3M additional parameters to achieve comparable performance to the fully fine-tuned Gatortron-base model, both surpassing 0.9 F1 score. Conclusion: Open-source instruction-tuned LLMs demonstrate impressive in-context learning capability in the mobility functioning classification task. The performance of these models can be further improved by continuing fine-tuning on a task-specific dataset.
△ Less
Submitted 15 December, 2023;
originally announced December 2023.
-
Leveraging deep active learning to identify low-resource mobility functioning information in public clinical notes
Authors:
Tuan-Dung Le,
Zhuqi Miao,
Samuel Alvarado,
Brittany Smith,
William Paiva,
Thanh Thieu
Abstract:
Function is increasingly recognized as an important indicator of whole-person health, although it receives little attention in clinical natural language processing research. We introduce the first public annotated dataset specifically on the Mobility domain of the International Classification of Functioning, Disability and Health (ICF), aiming to facilitate automatic extraction and analysis of fun…
▽ More
Function is increasingly recognized as an important indicator of whole-person health, although it receives little attention in clinical natural language processing research. We introduce the first public annotated dataset specifically on the Mobility domain of the International Classification of Functioning, Disability and Health (ICF), aiming to facilitate automatic extraction and analysis of functioning information from free-text clinical notes. We utilize the National NLP Clinical Challenges (n2c2) research dataset to construct a pool of candidate sentences using keyword expansion. Our active learning approach, using query-by-committee sampling weighted by density representativeness, selects informative sentences for human annotation. We train BERT and CRF models, and use predictions from these models to guide the selection of new sentences for subsequent annotation iterations. Our final dataset consists of 4,265 sentences with a total of 11,784 entities, including 5,511 Action entities, 5,328 Mobility entities, 306 Assistance entities, and 639 Quantification entities. The inter-annotator agreement (IAA), averaged over all entity types, is 0.72 for exact matching and 0.91 for partial matching. We also train and evaluate common BERT models and state-of-the-art Nested NER models. The best F1 scores are 0.84 for Action, 0.7 for Mobility, 0.62 for Assistance, and 0.71 for Quantification. Empirical results demonstrate promising potential of NER models to accurately extract mobility functioning information from clinical text. The public availability of our annotated dataset will facilitate further research to comprehensively capture functioning information in electronic health records (EHRs).
△ Less
Submitted 27 November, 2023;
originally announced November 2023.
-
Social human collective decision-making and its applications with brain network models
Authors:
Thoa Thieu,
Roderick Melnik
Abstract:
In this chapter, we consider probabilistic drift-diffusion models and Bayesian inference frameworks to address this issue, assisting better social human decision-making. We provide details of the models, as well as representative numerical examples, and discuss the decision-making process with a representative example of the escape route decision-making phenomena by further develo** the drift-di…
▽ More
In this chapter, we consider probabilistic drift-diffusion models and Bayesian inference frameworks to address this issue, assisting better social human decision-making. We provide details of the models, as well as representative numerical examples, and discuss the decision-making process with a representative example of the escape route decision-making phenomena by further develo** the drift-diffusion models and Bayesian inference frameworks. In the latter context, we also give a review of recent developments in human collective decision-making and its applications with brain network models. Furthermore, we provide illustrative numerical examples to discuss the role of neuromodulation, reinforcement learning in decision-making processes. Finally, we call attention to existing challenges, open problems, and promising approaches in studying social dynamics and collective human decision-making, including those arising from nonequilibrium considerations of the associated processes.
△ Less
Submitted 6 July, 2023;
originally announced July 2023.
-
Coupled stochastic systems of Skorokhod type: well-posedness of a mathematical model and its applications
Authors:
Thi Kim Thoa Thieu,
Adrian Muntean,
Roderick Melnik
Abstract:
Population dynamics with complex biological interactions, accounting for uncertainty quantification, is critical for many application areas. However, due to the complexity of biological systems, the mathematical formulation of the corresponding problems faces the challenge that the corresponding stochastic processes should, in most cases, be considered in bounded domains. We propose a model based…
▽ More
Population dynamics with complex biological interactions, accounting for uncertainty quantification, is critical for many application areas. However, due to the complexity of biological systems, the mathematical formulation of the corresponding problems faces the challenge that the corresponding stochastic processes should, in most cases, be considered in bounded domains. We propose a model based on a coupled system of reflecting Skorokhod-type stochastic differential equations with jump-like exit from a boundary. The setting describes the population dynamics of active and passive populations. As main working techniques, we use compactness methods and Skorokhod's representation of solutions to SDEs posed in bounded domains to prove the well-posedness of the system. This functional setting is a new point of view in the field of modelling and simulation of population dynamics. We provide the details of the model, as well as representative numerical examples, and discuss the applications of a Wilson-Cowan-type system, modelling the dynamics of two interacting populations of excitatory and inhibitory neurons. Furthermore, the presence of random input current, reflecting factors together with Poisson jumps, increases firing activity in neuronal systems.
△ Less
Submitted 16 June, 2022;
originally announced June 2022.
-
Effects of random inputs and short-term synaptic plasticity in a LIF conductance model for working memory applications
Authors:
Thi Kim Thoa Thieu,
Roderick Melnik
Abstract:
Working memory (WM) has been intensively used to enable the temporary storing of information for processing purposes, playing an important role in the execution of various cognitive tasks. Recent studies have shown that information in WM is not only maintained through persistent recurrent activity but also can be stored in activity-silent states such as in short-term synaptic plasticity (STSP). Mo…
▽ More
Working memory (WM) has been intensively used to enable the temporary storing of information for processing purposes, playing an important role in the execution of various cognitive tasks. Recent studies have shown that information in WM is not only maintained through persistent recurrent activity but also can be stored in activity-silent states such as in short-term synaptic plasticity (STSP). Motivated by important applications of the STSP mechanisms in WM, the main focus of the present work is on the analysis of the effects of random inputs on a leaky integrate-and-fire (LIF) synaptic conductance neuron under STSP. Furthermore, the irregularity of spike trains can carry the information about previous stimulation in a neuron. A LIF conductance neuron with multiple inputs and coefficient of variation (CV) of the inter-spike-interval (ISI) can bring an output decoded neuron. Our numerical results show that an increase in the standard deviations in the random input current and the random refractory period can lead to an increased irregularity of spike trains of the output neuron.
△ Less
Submitted 9 May, 2022;
originally announced May 2022.
-
Effects of noise on leaky integrate-and-fire neuron models for neuromorphic computing applications
Authors:
Thi Kim Thoa Thieu,
Roderick Melnik
Abstract:
Artificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), inspired by biological neurons enable a more realistic mimicry of the human brain. A large class of the problems from these domains is characterized by t…
▽ More
Artificial neural networks (ANNs) have been extensively used for the description of problems arising from biological systems and for constructing neuromorphic computing models. The third generation of ANNs, namely, spiking neural networks (SNNs), inspired by biological neurons enable a more realistic mimicry of the human brain. A large class of the problems from these domains is characterized by the necessity to deal with the combination of neurons, spikes and synapses via integrate-and-fire neuron models. Motivated by important applications of the integrate-and-fire of neurons in neuromorphic computing for bio-medical studies, the main focus of the present work is on the analysis of the effects of additive and multiplicative types of random input currents together with a random refractory period on a leaky integrate-and-fire (LIF) synaptic conductance neuron model. Our analysis is carried out via Langevin stochastic dynamics in a numerical setting describing a cell membrane potential. We provide the details of the model, as well as representative numerical examples, and discuss the effects of noise on the time evolution of the membrane potential as well as the spiking activities of neurons in the LIF synaptic conductance model scrutinized here. Furthermore, our numerical results demonstrate that the presence of a random refractory period in the LIF synaptic conductance system may substantially influence an increased irregularity of spike trains of the output neuron.
△ Less
Submitted 18 May, 2022; v1 submitted 18 February, 2022;
originally announced February 2022.
-
Modelling the behavior of human crowds as coupled active-passive dynamics of interacting particle systems
Authors:
Thi Kim Thoa Thieu,
Roderick Melnik
Abstract:
The modelling of human crowd behaviors offers many challenging questions to science in general. Specifically, the social human behavior consists of many physiological and psychological processes which are still largely unknown. To model reliably such human crowd systems with complex social interactions, stochastic tools play an important role for the setting of mathematical formulations of the pro…
▽ More
The modelling of human crowd behaviors offers many challenging questions to science in general. Specifically, the social human behavior consists of many physiological and psychological processes which are still largely unknown. To model reliably such human crowd systems with complex social interactions, stochastic tools play an important role for the setting of mathematical formulations of the problems. In this work, using the description based on an exclusion principle, we study a statistical-mechanics-based lattice gas model for active-passive population dynamics with an application to human crowd behaviors. We provide representative numerical examples for the evacuation dynamics of human crowds, where the main focus in our considerations is given to an interacting particle system of active and passive human groups. Furthermore, our numerical results show that the communication between active and passive humans strongly influences the evacuation time of the whole population even when the "faster-is-slower" phenomenon is taken into account. To provide an additional inside into the problem, a stationary state of our model is analyzed via current representations and heat map techniques. Finally, future extensions of the proposed models are discussed in the context of coupled data-driven modelling of human crowds and traffic flows, vital for the design strategies in develo** intelligent transportation systems.
△ Less
Submitted 16 October, 2023; v1 submitted 31 December, 2021;
originally announced January 2022.
-
Coupled effects of channels and synaptic dynamics in stochastic modelling of healthy and Parkinson's-disease-affected brains
Authors:
Thi Kim Thoa Thieu,
Roderick Melnik
Abstract:
Our brain is a complex information processing network in which the nervous system receives information from the environment to quickly react to incoming events or learns from experience to sharp our memory. In the nervous system, the brain states translate collective activities of neurons interconnected via synaptic connections. In this paper, we study coupled effects of channels and synaptic dyna…
▽ More
Our brain is a complex information processing network in which the nervous system receives information from the environment to quickly react to incoming events or learns from experience to sharp our memory. In the nervous system, the brain states translate collective activities of neurons interconnected via synaptic connections. In this paper, we study coupled effects of channels and synaptic dynamics under the stochastic influence of healthy brain cells with applications to Parkinson's disease (PD). In particular, we investigate the effects of random inputs in a subthalamic nucleus (STN) cell membrane potential model. The STN bursting phenomena and parkinsonian hypokinetic motor symptoms are closely connected, as electrical and chemical maneuvers modulating STN bursts are sufficient to ameliorate or mimic parkinsonian motor deficits. Deep brain stimulation (DBS) of the STN is an important surgical technique used in the treatment to improve PD symptoms. Our numerical results show that the random inputs strongly affect the spiking activities of the STN neuron not only in the case of healthy cells but also in the case of PD cells in the presence of DBS treatment. Specifically, the existence of a random refractory period together with random input current in the system may substantially influence an increased irregularity of spike trains of the output neurons.
△ Less
Submitted 16 June, 2022; v1 submitted 23 December, 2021;
originally announced December 2021.
-
Combining Coupled Skorokhod SDEs and Lattice Gas Frameworks for Multi-fidelity Modelling of Complex Behavioral Systems
Authors:
Thi Kim Thoa Thieu,
Roderick Melnik
Abstract:
To model reliably behavioral systems with complex bio-social interactions, accounting for uncertainty quantification, is critical for many application areas. However, in terms of the mathematical formulation of the corresponding problems, one of the major challenges is coming from the fact that corresponding stochastic processes should, in most cases, be considered in bounded domains, possibly wit…
▽ More
To model reliably behavioral systems with complex bio-social interactions, accounting for uncertainty quantification, is critical for many application areas. However, in terms of the mathematical formulation of the corresponding problems, one of the major challenges is coming from the fact that corresponding stochastic processes should, in most cases, be considered in bounded domains, possibly with obstacles. This has been known for a long time and yet, very little has been done for the quantification of uncertainties in modelling complex behavioral systems described by such stochastic processes. In this paper, we address this challenge by considering a coupled system of Skorokhod-type stochastic differential equations (SDEs) describing interactions between active and passive participants of a mixed-population group. In develo** a multi-fidelity modelling methodology for such behavioral systems, we combine low- and high-fidelity results obtained from (a) the solution of the underlying coupled system of SDEs and (b) simulations with a statistical-mechanics-based lattice gas model, where we employ a kinetic Monte Carlo procedure. Furthermore, we provide representative numerical examples of healthcare systems, subject to an epidemic, where the main focus in our considerations is given to an interacting particle system of asymptomatic and susceptible populations.
△ Less
Submitted 21 June, 2021;
originally announced June 2021.
-
Reflecting stochastic dynamics of active-passive populations with applications in operations research and neuroscience
Authors:
Thi Kim Thoa Thieu,
Roderick Melnik
Abstract:
Stochastic dynamic models have been extensively used for the description of processes with uncertainties arising in the operations research, behavioral sciences, and many other application areas. A large class of the problems from these domains is characterized by the necessity to deal with several distinct groups of populations, which are usually labeled as "active" and "passive". Motivated by im…
▽ More
Stochastic dynamic models have been extensively used for the description of processes with uncertainties arising in the operations research, behavioral sciences, and many other application areas. A large class of the problems from these domains is characterized by the necessity to deal with several distinct groups of populations, which are usually labeled as "active" and "passive". Motivated by important applications of queueing networks and neuroscience, the main focus of the present work is on the analysis of reflecting stochastic dynamics of such mixed populations. We develop a general mathematical modeling framework to describe the reflecting stochastic dynamics for active-passive populations. The analysis of this model is carried out via a combination of low- and high-delity results obtained from the solution of the underlying coupled system of SDEs and from the simulations with a statistical-mechanics-based lattice gas model, where we employ a kinetic Monte Carlo procedure. We provide details of the queueing theory and neuronal models and discuss a relationship between reflecting SDEs and a model of queueing theory via a limit theorem. Furthermore, we present several representative numerical examples, and discuss an intrinsic interconnection between active and passive particles in the underlying stochastic process. Finally, possible extensions of the proposed methodology have been highlighted.
△ Less
Submitted 31 May, 2021; v1 submitted 13 February, 2021;
originally announced February 2021.
-
Well-posedness of a coupled system of Skorohod-like stochastic differential equations
Authors:
Thi Kim Thoa Thieu,
Adrian Muntean
Abstract:
We study the well-posedness of a coupled system of Skorohod-like stochastic differential equations with reflecting boundary condition. The setting describes the evacuation dynamics of a mixed crowd composed of both active and passive pedestrians moving through a domain with obstacles, fire and smoke. As main working techniques, we use compactness methods and the Skorohod's representation of soluti…
▽ More
We study the well-posedness of a coupled system of Skorohod-like stochastic differential equations with reflecting boundary condition. The setting describes the evacuation dynamics of a mixed crowd composed of both active and passive pedestrians moving through a domain with obstacles, fire and smoke. As main working techniques, we use compactness methods and the Skorohod's representation of solutions to SDEs posed in bounded domains. This functional setting is a new point of view in the field of modeling and simulation pedestrian dynamics. The main challenge is to handle the coupling in the model equations together with the multiple-connectedness of the domain and the pedestrian-obstacle interaction.
△ Less
Submitted 6 February, 2021; v1 submitted 30 May, 2020;
originally announced June 2020.
-
When diffusion faces drift: consequences of exclusion processes for bi--directional pedestrian flows
Authors:
Emilio N. M. Cirillo,
Matteo Colangeli,
Adrian Muntean,
T. K. Thoa Thieu
Abstract:
Stochastic particle--based models are useful tools for describing the collective movement of large crowds of pedestrians in crowded confined environments. Using descriptions based on the simple exclusion process, two populations of particles, mimicking pedestrians walking in a built environment, enter a room from two opposite sides. One population is passive -- being unaware of the local environme…
▽ More
Stochastic particle--based models are useful tools for describing the collective movement of large crowds of pedestrians in crowded confined environments. Using descriptions based on the simple exclusion process, two populations of particles, mimicking pedestrians walking in a built environment, enter a room from two opposite sides. One population is passive -- being unaware of the local environment; particles belonging to this group perform a symmetric random walk. The other population has information on the local geometry in the sense that as soon as particles enter a visibility zone, a drift activates them. Their self-propulsion leads them towards the exit. This second type of species is referred here as active. The assumed crowdedness corresponds to a near--jammed scenario. The main question we ask in this paper is: Can we induce modifications of the dynamics of the active particles to improve the outgoing current of the passive particles? To address this question, we compute occupation number profiles and currents for both populations in selected parameter ranges. Besides observing the more classical faster--is--slower effect, new features appear as prominent like the non--monotonicity of currents, self--induced phase separation within the active population, as well as acceleration of passive particles for large--drift regimes of active particles.
△ Less
Submitted 2 March, 2020;
originally announced March 2020.
-
Uniqueness and stability with respect to parameters of solutions to a fluid-like driven system for active-passive pedestrian dynamics
Authors:
T. K. Thoa Thieu,
Matteo Colangeli,
Adrian Muntean
Abstract:
We study a system of parabolic equations consisting of a double nonlinear parabolic equations of Forchheimer type coupled with a semilinear parabolic equations. The system describes a fluid-like driven system for active-passive pedestrian dynamics. The structure of the nonlinearity of the coupling allows us to prove the uniqueness of solutions. We provide also stability estimates of solutions with…
▽ More
We study a system of parabolic equations consisting of a double nonlinear parabolic equations of Forchheimer type coupled with a semilinear parabolic equations. The system describes a fluid-like driven system for active-passive pedestrian dynamics. The structure of the nonlinearity of the coupling allows us to prove the uniqueness of solutions. We provide also stability estimates of solutions with respect to selected parameters.
△ Less
Submitted 27 December, 2019;
originally announced December 2019.
-
Weak solvability a fluid-like driven system for active-passive pedestrian dynamics
Authors:
T. K. Thoa Thieu,
Matteo Colangeli,
Adrian Muntean
Abstract:
We study the question of weak solvability for a nonlinear coupled parabolic system that models the evolution of a complex pedestrian flow. The main feature is that the flow is composed of a mix of densities of active and passive pedestrians that are moving with different velocities. We rely on special energy estimates and on the use a Schauder's fixed point argument to tackle the existence of solu…
▽ More
We study the question of weak solvability for a nonlinear coupled parabolic system that models the evolution of a complex pedestrian flow. The main feature is that the flow is composed of a mix of densities of active and passive pedestrians that are moving with different velocities. We rely on special energy estimates and on the use a Schauder's fixed point argument to tackle the existence of solutions to our evolution problem.
△ Less
Submitted 11 October, 2019;
originally announced October 2019.
-
A lattice model for active--passive pedestrian dynamics: a quest for drafting effects
Authors:
Emilio N. M. Cirillo,
Matteo Colangeli,
Adrian Muntean,
T. K. Thoa Thieu
Abstract:
We study the pedestrian escape from an obscure corridor using a lattice gas model with two species of particles. One species, called passive, performs a symmetric random walk on the lattice, whereas the second species, called active, is subject to a drift guiding the particles towards the exit. The drift mimics the awareness of some pedestrians of the geometry of the corridor and of the location o…
▽ More
We study the pedestrian escape from an obscure corridor using a lattice gas model with two species of particles. One species, called passive, performs a symmetric random walk on the lattice, whereas the second species, called active, is subject to a drift guiding the particles towards the exit. The drift mimics the awareness of some pedestrians of the geometry of the corridor and of the location of the exit. We provide numerical evidence that, in spite of the hard core interaction between particles -- namely, there can be at most one particle of any species per site, -- adding a fraction of active particles in the system enhances the evacuation rate of all particles from the corridor. A similar effect is also observed when looking at the outgoing particle flux, when the system is in contact with an external particle reservoir that induces the onset of a steady state. We interpret this phenomenon as a discrete space counterpart of the drafting effect typically observed in a continuum set--up as the aerodynamic drag experienced by pelotons of competing cyclists.
△ Less
Submitted 19 July, 2019;
originally announced July 2019.
-
Modelling interactions between active and passive agents moving through heterogeneous environments
Authors:
Matteo Colangeli,
Adrian Muntean,
Omar Richardson,
Thoa Thieu
Abstract:
We study the dynamics of interacting agents from two distinct inter-mixed populations: One population includes active agents that follow a predetermined velocity field, while the second population contains exclusively passive agents, i.e. agents that have no preferred direction of motion. The orientation of their local velocity is affected by repulsive interactions with the neighboring agents and…
▽ More
We study the dynamics of interacting agents from two distinct inter-mixed populations: One population includes active agents that follow a predetermined velocity field, while the second population contains exclusively passive agents, i.e. agents that have no preferred direction of motion. The orientation of their local velocity is affected by repulsive interactions with the neighboring agents and environment. We present two models that allow for a qualitative analysis of these mixed systems. We show that the residence times of this type of systems containing mixed populations is strongly affected by the interplay between these two populations. After showing our modeling and simulation results, we conclude with a couple of mathematical aspects concerning the well-posedness of our models.
△ Less
Submitted 5 June, 2018; v1 submitted 23 May, 2018;
originally announced May 2018.