-
Modelling the Raft Distributed Consensus Protocol in mCRL2
Authors:
Parth Bora,
Pham Duc Minh,
Tim A. C. Willemse
Abstract:
The consensus problem is a fundamental problem in distributed systems. It involves a set of actors, or entities, that need to agree on some values or decisions. The Raft algorithm is a solution to the consensus problem that has gained widespread popularity as an easy-to-understand and implement alternative to Lamport's Paxos algorithm. In this paper we discuss a formalisation of the Raft algorithm…
▽ More
The consensus problem is a fundamental problem in distributed systems. It involves a set of actors, or entities, that need to agree on some values or decisions. The Raft algorithm is a solution to the consensus problem that has gained widespread popularity as an easy-to-understand and implement alternative to Lamport's Paxos algorithm. In this paper we discuss a formalisation of the Raft algorithm and its associated correctness properties in the mCRL2 specification language.
△ Less
Submitted 27 March, 2024;
originally announced March 2024.
-
Overview of Class Activation Maps for Visualization Explainability
Authors:
Anh Pham Thi Minh
Abstract:
Recent research in deep learning methodology has led to a variety of complex modelling techniques in computer vision (CV) that reach or even outperform human performance. Although these black-box deep learning models have obtained astounding results, they are limited in their interpretability and transparency which are critical to take learning machines to the next step to include them in sensitiv…
▽ More
Recent research in deep learning methodology has led to a variety of complex modelling techniques in computer vision (CV) that reach or even outperform human performance. Although these black-box deep learning models have obtained astounding results, they are limited in their interpretability and transparency which are critical to take learning machines to the next step to include them in sensitive decision-support systems involving human supervision. Hence, the development of explainable techniques for computer vision (XCV) has recently attracted increasing attention. In the realm of XCV, Class Activation Maps (CAMs) have become widely recognized and utilized for enhancing interpretability and insights into the decision-making process of deep learning models. This work presents a comprehensive overview of the evolution of Class Activation Map methods over time. It also explores the metrics used for evaluating CAMs and introduces auxiliary techniques to improve the saliency of these methods. The overview concludes by proposing potential avenues for future research in this evolving field.
△ Less
Submitted 25 September, 2023;
originally announced September 2023.
-
Using network metrics to explore the community structure that underlies movement patterns
Authors:
Anh Pham Thi Minh,
Abhishek Kumar Singh,
Soumya Snigdha Kundu
Abstract:
This work aims to explore the community structure of Santiago de Chile by analyzing the movement patterns of its residents. We use a dataset containing the approximate locations of home and work places for a subset of anonymized residents to construct a network that represents the movement patterns within the city. Through the analysis of this network, we aim to identify the communities or sub-cit…
▽ More
This work aims to explore the community structure of Santiago de Chile by analyzing the movement patterns of its residents. We use a dataset containing the approximate locations of home and work places for a subset of anonymized residents to construct a network that represents the movement patterns within the city. Through the analysis of this network, we aim to identify the communities or sub-cities that exist within Santiago de Chile and gain insights into the factors that drive the spatial organization of the city. We employ modularity optimization algorithms and clustering techniques to identify the communities within the network. Our results present that the novelty of combining community detection algorithms with segregation tools provides new insights to further the understanding of the complex geography of segregation during working hours.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
PRE: Vision-Language Prompt Learning with Reparameterization Encoder
Authors:
Anh Pham Thi Minh,
An Duc Nguyen,
Georgios Tzimiropoulos
Abstract:
Large pre-trained vision-language models such as CLIP have demonstrated great potential in zero-shot transferability to downstream tasks. However, to attain optimal performance, the manual selection of prompts is necessary to improve alignment between the downstream image distribution and the textual class descriptions. This manual prompt engineering is the major challenge for deploying such model…
▽ More
Large pre-trained vision-language models such as CLIP have demonstrated great potential in zero-shot transferability to downstream tasks. However, to attain optimal performance, the manual selection of prompts is necessary to improve alignment between the downstream image distribution and the textual class descriptions. This manual prompt engineering is the major challenge for deploying such models in practice since it requires domain expertise and is extremely time-consuming. To avoid non-trivial prompt engineering, recent work Context Optimization (CoOp) introduced the concept of prompt learning to the vision domain using learnable textual tokens. While CoOp can achieve substantial improvements over manual prompts, its learned context is worse generalizable to wider unseen classes within the same dataset. In this work, we present Prompt Learning with Reparameterization Encoder (PRE) - a simple and efficient method that enhances the generalization ability of the learnable prompt to unseen classes while maintaining the capacity to learn Base classes. Instead of directly optimizing the prompts, PRE employs a prompt encoder to reparameterize the input prompt embeddings, enhancing the exploration of task-specific knowledge from few-shot samples. Experiments and extensive ablation studies on 8 benchmarks demonstrate that our approach is an efficient method for prompt learning. Specifically, PRE achieves a notable enhancement of 5.60% in average accuracy on New classes and 3% in Harmonic mean compared to CoOp in the 16-shot setting, all achieved within a good training time.
△ Less
Submitted 6 November, 2023; v1 submitted 14 September, 2023;
originally announced September 2023.
-
Active Label Refinement for Semantic Segmentation of Satellite Images
Authors:
Tuan Pham Minh,
Jayan Wijesingha,
Daniel Kottke,
Marek Herde,
Denis Huseljic,
Bernhard Sick,
Michael Wachendorf,
Thomas Esch
Abstract:
Remote sensing through semantic segmentation of satellite images contributes to the understanding and utilisation of the earth's surface. For this purpose, semantic segmentation networks are typically trained on large sets of labelled satellite images. However, obtaining expert labels for these images is costly. Therefore, we propose to rely on a low-cost approach, e.g. crowdsourcing or pretrained…
▽ More
Remote sensing through semantic segmentation of satellite images contributes to the understanding and utilisation of the earth's surface. For this purpose, semantic segmentation networks are typically trained on large sets of labelled satellite images. However, obtaining expert labels for these images is costly. Therefore, we propose to rely on a low-cost approach, e.g. crowdsourcing or pretrained networks, to label the images in the first step. Since these initial labels are partially erroneous, we use active learning strategies to cost-efficiently refine the labels in the second step. We evaluate the active learning strategies using satellite images of Bengaluru in India, labelled with land cover and land use labels. Our experimental results suggest that an active label refinement to improve the semantic segmentation network's performance is beneficial.
△ Less
Submitted 12 September, 2023;
originally announced September 2023.
-
ViMQ: A Vietnamese Medical Question Dataset for Healthcare Dialogue System Development
Authors:
Ta Duc Huy,
Nguyen Anh Tu,
Tran Hoang Vu,
Nguyen Phuc Minh,
Nguyen Phan,
Trung H. Bui,
Steven Q. H. Truong
Abstract:
Existing medical text datasets usually take the form of question and answer pairs that support the task of natural language generation, but lacking the composite annotations of the medical terms. In this study, we publish a Vietnamese dataset of medical questions from patients with sentence-level and entity-level annotations for the Intent Classification and Named Entity Recognition tasks. The tag…
▽ More
Existing medical text datasets usually take the form of question and answer pairs that support the task of natural language generation, but lacking the composite annotations of the medical terms. In this study, we publish a Vietnamese dataset of medical questions from patients with sentence-level and entity-level annotations for the Intent Classification and Named Entity Recognition tasks. The tag sets for two tasks are in medical domain and can facilitate the development of task-oriented healthcare chatbots with better comprehension of queries from patients. We train baseline models for the two tasks and propose a simple self-supervised training strategy with span-noise modelling that substantially improves the performance. Dataset and code will be published at https://github.com/tadeephuy/ViMQ
△ Less
Submitted 27 April, 2023;
originally announced April 2023.
-
Develo** a Machine-Learning Algorithm to Diagnose Age-Related Macular Degeneration
Authors:
Ananya Dua,
Pham Hung Minh,
Sajid Fahmid,
Shikhar Gupta,
Sophia Zheng,
Vanessa Moyo,
Yanran Elisa Xue
Abstract:
Today, more than 12 million people over the age of 40 suffer from ocular diseases. Most commonly, older patients are susceptible to age related macular degeneration, an eye disease that causes blurring of the central vision due to the deterioration of the retina. The former can only be detected through complex and expensive imaging software, markedly a visual field test; this leaves a significant…
▽ More
Today, more than 12 million people over the age of 40 suffer from ocular diseases. Most commonly, older patients are susceptible to age related macular degeneration, an eye disease that causes blurring of the central vision due to the deterioration of the retina. The former can only be detected through complex and expensive imaging software, markedly a visual field test; this leaves a significant population with untreated eye disease and holds them at risk for complete vision loss. The use of machine learning algorithms has been proposed for treating eye disease. However, the development of these models is limited by a lack of understanding regarding appropriate model and training parameters to maximize model performance. In our study, we address these points by generating 6 models, each with a learning rate of 1 * 10^n where n is 0, -1, -2, ... -6, and calculated a f1 score for each of the models. Our analysis shows that sample imbalance is a key challenge in training of machine learning models and can result in deceptive improvements in training cost which does not translate to true improvements in model predictive performance. Considering the wide ranging impact of the disease and its adverse effects, we developed a machine learning algorithm to treat the same. We trained our model on varying eye disease datasets consisting of over 5000 patients, and the pictures of their infected eyes. In the future, we hope this model is used extensively, especially in areas that are under-resourced, to better diagnose eye disease and improve well being for humanity.
△ Less
Submitted 28 January, 2022;
originally announced January 2022.
-
ACE: A Novel Approach for the Statistical Analysis of Pairwise Connectivity
Authors:
Krempl,
Georg,
Kottke,
Daniel,
Pham Minh,
Tuan
Abstract:
Analysing correlations between streams of events is an important problem. It arises for example in Neurosciences, when the connectivity of neurons should be inferred from spike trains that record neurons' individual spiking activity. While recently some approaches for inferring delayed synaptic connections have been proposed, they are limited in the types of connectivities and delays they are able…
▽ More
Analysing correlations between streams of events is an important problem. It arises for example in Neurosciences, when the connectivity of neurons should be inferred from spike trains that record neurons' individual spiking activity. While recently some approaches for inferring delayed synaptic connections have been proposed, they are limited in the types of connectivities and delays they are able to handle, or require computation-intensive procedures. This paper proposes a faster and more flexible approach for analysing such delayed correlated activity: a statistical approach for the Analysis of Connectivity in spiking Events (ACE), based on the idea of hypothesis testing. It first computes for any pair of a source and a target neuron the inter-spike delays between subsequent source- and target-spikes. Then, it derives a null model for the distribution of inter-spike delays for \emph{uncorrelated}~neurons. Finally, it compares the observed distribution of inter-spike delays to this null model and infers pairwise connectivity based on the Pearson's Chi-squared test statistic. Thus, ACE is capable to detect connections with a priori unknown, non-discrete (and potentially large) inter-spike delays, which might vary between pairs of neurons. Since ACE works incrementally, it has potential for being used in online processing. In our experiments, we visualise the advantages of ACE in varying experimental scenarios (except for one special case) and in a state-of-the-art dataset which has been generated for neuro-scientific research under most realistic conditions.
△ Less
Submitted 9 August, 2021;
originally announced August 2021.
-
Probabilistic Active Learning for Active Class Selection
Authors:
Daniel Kottke,
Georg Krempl,
Marianne Stecklina,
Cornelius Styp von Rekowski,
Tim Sabsch,
Tuan Pham Minh,
Matthias Deliano,
Myra Spiliopoulou,
Bernhard Sick
Abstract:
In machine learning, active class selection (ACS) algorithms aim to actively select a class and ask the oracle to provide an instance for that class to optimize a classifier's performance while minimizing the number of requests. In this paper, we propose a new algorithm (PAL-ACS) that transforms the ACS problem into an active learning task by introducing pseudo instances. These are used to estimat…
▽ More
In machine learning, active class selection (ACS) algorithms aim to actively select a class and ask the oracle to provide an instance for that class to optimize a classifier's performance while minimizing the number of requests. In this paper, we propose a new algorithm (PAL-ACS) that transforms the ACS problem into an active learning task by introducing pseudo instances. These are used to estimate the usefulness of an upcoming instance for each class using the performance gain model from probabilistic active learning. Our experimental evaluation (on synthetic and real data) shows the advantages of our algorithm compared to state-of-the-art algorithms. It effectively prefers the sampling of difficult classes and thereby improves the classification performance.
△ Less
Submitted 9 August, 2021;
originally announced August 2021.
-
Multimodal Fusion with BERT and Attention Mechanism for Fake News Detection
Authors:
Nguyen Manh Duc Tuan,
Pham Quang Nhat Minh
Abstract:
Fake news detection is an important task for increasing the credibility of information on the media since fake news is constantly spreading on social media every day and it is a very serious concern in our society. Fake news is usually created by manipulating images, texts, and videos. In this paper, we present a novel method for detecting fake news by fusing multimodal features derived from textu…
▽ More
Fake news detection is an important task for increasing the credibility of information on the media since fake news is constantly spreading on social media every day and it is a very serious concern in our society. Fake news is usually created by manipulating images, texts, and videos. In this paper, we present a novel method for detecting fake news by fusing multimodal features derived from textual and visual data. Specifically, we used a pre-trained BERT model to learn text features and a VGG-19 model pre-trained on the ImageNet dataset to extract image features. We proposed a scale-dot product attention mechanism to capture the relationship between text features and visual features. Experimental results showed that our approach performs better than the current state-of-the-art method on a public Twitter dataset by 3.1% accuracy.
△ Less
Submitted 27 April, 2021; v1 submitted 23 April, 2021;
originally announced April 2021.
-
Efficient Low-Latency Dynamic Licensing for Deep Neural Network Deployment on Edge Devices
Authors:
Toan Pham Van,
Ngoc N. Tran,
Hoang Pham Minh,
Tam Nguyen Minh,
Thanh Ta Minh
Abstract:
Along with the rapid development in the field of artificial intelligence, especially deep learning, deep neural network applications are becoming more and more popular in reality. To be able to withstand the heavy load from mainstream users, deployment techniques are essential in bringing neural network models from research to production. Among the two popular computing topologies for deploying ne…
▽ More
Along with the rapid development in the field of artificial intelligence, especially deep learning, deep neural network applications are becoming more and more popular in reality. To be able to withstand the heavy load from mainstream users, deployment techniques are essential in bringing neural network models from research to production. Among the two popular computing topologies for deploying neural network models in production are cloud-computing and edge-computing. Recent advances in communication technologies, along with the great increase in the number of mobile devices, has made edge-computing gradually become an inevitable trend. In this paper, we propose an architecture to solve deploying and processing deep neural networks on edge-devices by leveraging their synergy with the cloud and the access-control mechanisms of the database. Adopting this architecture allows low-latency DNN model updates on devices. At the same time, with only one model deployed, we can easily make different versions of it by setting access permissions on the model weights. This method allows for dynamic model licensing, which benefits commercial applications.
△ Less
Submitted 24 February, 2021;
originally announced February 2021.
-
An Empirical Study of Using Pre-trained BERT Models for Vietnamese Relation Extraction Task at VLSP 2020
Authors:
Pham Quang Nhat Minh
Abstract:
In this paper, we present an empirical study of using pre-trained BERT models for the relation extraction task at the VLSP 2020 Evaluation Campaign. We applied two state-of-the-art BERT-based models: R-BERT and BERT model with entity starts. For each model, we compared two pre-trained BERT models: FPTAI/vibert and NlpHUST/vibert4news. We found that NlpHUST/vibert4news model significantly outperfor…
▽ More
In this paper, we present an empirical study of using pre-trained BERT models for the relation extraction task at the VLSP 2020 Evaluation Campaign. We applied two state-of-the-art BERT-based models: R-BERT and BERT model with entity starts. For each model, we compared two pre-trained BERT models: FPTAI/vibert and NlpHUST/vibert4news. We found that NlpHUST/vibert4news model significantly outperforms FPTAI/vibert for the Vietnamese relation extraction task. Finally, we proposed an ensemble model that combines R-BERT and BERT with entity starts. Our proposed ensemble model slightly improved against two single models on the development data and the test data provided by the task organizers.
△ Less
Submitted 28 January, 2021; v1 submitted 18 December, 2020;
originally announced December 2020.
-
ReINTEL Challenge 2020: A Multimodal Ensemble Model for Detecting Unreliable Information on Vietnamese SNS
Authors:
Nguyen Manh Duc Tuan,
Pham Quang Nhat Minh
Abstract:
In this paper, we present our methods for unrealiable information identification task at VLSP 2020 ReINTEL Challenge. The task is to classify a piece of information into reliable or unreliable category. We propose a novel multimodal ensemble model which combines two multimodal models to solve the task. In each multimodal model, we combined feature representations acquired from three different data…
▽ More
In this paper, we present our methods for unrealiable information identification task at VLSP 2020 ReINTEL Challenge. The task is to classify a piece of information into reliable or unreliable category. We propose a novel multimodal ensemble model which combines two multimodal models to solve the task. In each multimodal model, we combined feature representations acquired from three different data types: texts, images, and metadata. Multimodal features are derived from three neural networks and fused for classification. Experimental results showed that our proposed multimodal ensemble model improved against single models in term of ROC AUC score. We obtained 0.9445 AUC score on the private test of the challenge.
△ Less
Submitted 18 December, 2020;
originally announced December 2020.
-
Weakly-Supervised Neural Response Selection from an Ensemble of Task-Specialised Dialogue Agents
Authors:
Asir Saeed,
Khai Mai,
Pham Minh,
Nguyen Tuan Duc,
Danushka Bollegala
Abstract:
Dialogue engines that incorporate different types of agents to converse with humans are popular.
However, conversations are dynamic in the sense that a selected response will change the conversation on-the-fly, influencing the subsequent utterances in the conversation, which makes the response selection a challenging problem.
We model the problem of selecting the best response from a set of re…
▽ More
Dialogue engines that incorporate different types of agents to converse with humans are popular.
However, conversations are dynamic in the sense that a selected response will change the conversation on-the-fly, influencing the subsequent utterances in the conversation, which makes the response selection a challenging problem.
We model the problem of selecting the best response from a set of responses generated by a heterogeneous set of dialogue agents by taking into account the conversational history, and propose a \emph{Neural Response Selection} method.
The proposed method is trained to predict a coherent set of responses within a single conversation, considering its own predictions via a curriculum training mechanism.
Our experimental results show that the proposed method can accurately select the most appropriate responses, thereby significantly improving the user experience in dialogue systems.
△ Less
Submitted 6 May, 2020;
originally announced May 2020.
-
A Feature-Based Model for Nested Named-Entity Recognition at VLSP-2018 NER Evaluation Campaign
Authors:
Pham Quang Nhat Minh
Abstract:
In this report, we describe our participant named-entity recognition system at VLSP 2018 evaluation campaign. We formalized the task as a sequence labeling problem using BIO encoding scheme. We applied a feature-based model which combines word, word-shape features, Brown-cluster-based features, and word-embedding-based features. We compare several methods to deal with nested entities in the datase…
▽ More
In this report, we describe our participant named-entity recognition system at VLSP 2018 evaluation campaign. We formalized the task as a sequence labeling problem using BIO encoding scheme. We applied a feature-based model which combines word, word-shape features, Brown-cluster-based features, and word-embedding-based features. We compare several methods to deal with nested entities in the dataset. We showed that combining tags of entities at all levels for training a sequence labeling model (joint-tag model) improved the accuracy of nested named-entity recognition.
△ Less
Submitted 22 March, 2018;
originally announced March 2018.
-
A Feature-Rich Vietnamese Named-Entity Recognition Model
Authors:
Pham Quang Nhat Minh
Abstract:
In this paper, we present a feature-based named-entity recognition (NER) model that achieves the start-of-the-art accuracy for Vietnamese language. We combine word, word-shape features, PoS, chunk, Brown-cluster-based features, and word-embedding-based features in the Conditional Random Fields (CRF) model. We also explore the effects of word segmentation, PoS tagging, and chunking results of many…
▽ More
In this paper, we present a feature-based named-entity recognition (NER) model that achieves the start-of-the-art accuracy for Vietnamese language. We combine word, word-shape features, PoS, chunk, Brown-cluster-based features, and word-embedding-based features in the Conditional Random Fields (CRF) model. We also explore the effects of word segmentation, PoS tagging, and chunking results of many popular Vietnamese NLP toolkits on the accuracy of the proposed feature-based NER model. Up to now, our work is the first work that systematically performs an extrinsic evaluation of basic Vietnamese NLP toolkits on the downstream NER task. Experimental results show that while automatically-generated word segmentation is useful, PoS and chunking information generated by Vietnamese NLP tools does not show their benefits for the proposed feature-based NER model.
△ Less
Submitted 12 March, 2018;
originally announced March 2018.
-
Discrete Bayesian Networks: The Exact Posterior Marginal Distributions
Authors:
Do Le Paul Minh
Abstract:
In a Bayesian network, we wish to evaluate the marginal probability of a query variable, which may be conditioned on the observed values of some evidence variables. Here we first present our "border algorithm," which converts a BN into a directed chain. For the polytrees, we then present in details, with some modifications and within the border algorithm framework, the "revised polytree algorithm"…
▽ More
In a Bayesian network, we wish to evaluate the marginal probability of a query variable, which may be conditioned on the observed values of some evidence variables. Here we first present our "border algorithm," which converts a BN into a directed chain. For the polytrees, we then present in details, with some modifications and within the border algorithm framework, the "revised polytree algorithm" by Peot & Shachter (1991). Finally, we present our "parentless polytree method," which, coupled with the border algorithm, converts any Bayesian network into a polytree, rendering the complexity of our inferences independent of the size of network, and linear with the number of its evidence and query variables. All quantities in this paper have probabilistic interpretations.
△ Less
Submitted 23 November, 2014;
originally announced November 2014.
-
Exploiting Direct and Indirect Information for Friend Suggestion in ZingMe
Authors:
Kien Duy Nguyen,
Tuan Pham Minh,
Quang Nhat Nguyen,
Thanh Trung Nguyen
Abstract:
Friend suggestion is a fundamental problem in social networks with the goal of assisting users in creating more relationships, and thereby enhances interest of users to the social networks. This problem is often considered to be the link prediction problem in the network. ZingMe is one of the largest social networks in Vietnam. In this paper, we analyze the current approach for the friend suggesti…
▽ More
Friend suggestion is a fundamental problem in social networks with the goal of assisting users in creating more relationships, and thereby enhances interest of users to the social networks. This problem is often considered to be the link prediction problem in the network. ZingMe is one of the largest social networks in Vietnam. In this paper, we analyze the current approach for the friend suggestion problem in ZingMe, showing its limitations and disadvantages. We propose a new efficient approach for friend suggestion that uses information from the network structure, attributes and interactions of users to create resources for the evaluation of friend connection amongst users. Friend connection is evaluated exploiting both direct communication between the users and information from other ones in the network. The proposed approach has been implemented in a new system version of ZingMe. We conducted experiments, exploiting a dataset derived from the users' real use of ZingMe, to compare the newly proposed approach to the current approach and some well-known ones for the accuracy of friend suggestion. The experimental results show that the newly proposed approach outperforms the current one, i.e., by an increase of 7% to 98% on average in the friend suggestion accuracy. The proposed approach also outperforms other ones for users who have a small number of friends with improvements from 20% to 85% on average. In this paper, we also discuss a number of open issues and possible improvements for the proposed approach.
△ Less
Submitted 15 November, 2013;
originally announced November 2013.
