-
A Parameter-free Adaptive Resonance Theory-based Topological Clustering Algorithm Capable of Continual Learning
Authors:
Naoki Masuyama,
Takanori Takebayashi,
Yusuke Nojima,
Chu Kiong Loo,
Hisao Ishibuchi,
Stefan Wermter
Abstract:
In general, a similarity threshold (i.e., a vigilance parameter) for a node learning process in Adaptive Resonance Theory (ART)-based algorithms has a significant impact on clustering performance. In addition, an edge deletion threshold in a topological clustering algorithm plays an important role in adaptively generating well-separated clusters during a self-organizing process. In this paper, we…
▽ More
In general, a similarity threshold (i.e., a vigilance parameter) for a node learning process in Adaptive Resonance Theory (ART)-based algorithms has a significant impact on clustering performance. In addition, an edge deletion threshold in a topological clustering algorithm plays an important role in adaptively generating well-separated clusters during a self-organizing process. In this paper, we propose a new parameter-free ART-based topological clustering algorithm capable of continual learning by introducing parameter estimation methods. Experimental results with synthetic and real-world datasets show that the proposed algorithm has superior clustering performance to the state-of-the-art clustering algorithms without any parameter pre-specifications.
△ Less
Submitted 2 May, 2023; v1 submitted 30 April, 2023;
originally announced May 2023.
-
An Exploratory Study on the Usage and Readability of Messages Within Assertion Methods of Test Cases
Authors:
Taryn Takebayashi,
Anthony Peruma,
Mohamed Wiem Mkaouer,
Christian D. Newman
Abstract:
Unit testing is a vital part of the software development process and involves developers writing code to verify or assert production code. Furthermore, to help comprehend the test case and troubleshoot issues, developers have the option to provide a message that explains the reason for the assertion failure. In this exploratory empirical study, we examine the characteristics of assertion messages…
▽ More
Unit testing is a vital part of the software development process and involves developers writing code to verify or assert production code. Furthermore, to help comprehend the test case and troubleshoot issues, developers have the option to provide a message that explains the reason for the assertion failure. In this exploratory empirical study, we examine the characteristics of assertion messages contained in the test methods in 20 open-source Java systems. Our findings show that while developers rarely utilize the option of supplying a message, those who do, either compose it of only string literals, identifiers, or a combination of both types. Using standard English readability measuring techniques, we observe that a beginner's knowledge of English is required to understand messages containing only identifiers, while a 4th-grade education level is required to understand messages composed of string literals. We also discuss shortcomings with using such readability measuring techniques and common anti-patterns in assert message construction. We envision our results incorporated into code quality tools that appraise the understandability of assertion messages.
△ Less
Submitted 28 February, 2023;
originally announced March 2023.
-
Crowdsourcing Evaluation of Saliency-based XAI Methods
Authors:
Xiaotian Lu,
Arseny Tolmachev,
Tatsuya Yamamoto,
Koh Takeuchi,
Seiji Okajima,
Tomoyoshi Takebayashi,
Koji Maruhashi,
Hisashi Kashima
Abstract:
Understanding the reasons behind the predictions made by deep neural networks is critical for gaining human trust in many important applications, which is reflected in the increasing demand for explainability in AI (XAI) in recent years. Saliency-based feature attribution methods, which highlight important parts of images that contribute to decisions by classifiers, are often used as XAI methods,…
▽ More
Understanding the reasons behind the predictions made by deep neural networks is critical for gaining human trust in many important applications, which is reflected in the increasing demand for explainability in AI (XAI) in recent years. Saliency-based feature attribution methods, which highlight important parts of images that contribute to decisions by classifiers, are often used as XAI methods, especially in the field of computer vision. In order to compare various saliency-based XAI methods quantitatively, several approaches for automated evaluation schemes have been proposed; however, there is no guarantee that such automated evaluation metrics correctly evaluate explainability, and a high rating by an automated evaluation scheme does not necessarily mean a high explainability for humans. In this study, instead of the automated evaluation, we propose a new human-based evaluation scheme using crowdsourcing to evaluate XAI methods. Our method is inspired by a human computation game, "Peek-a-boom", and can efficiently compare different XAI methods by exploiting the power of crowds. We evaluate the saliency maps of various XAI methods on two datasets with automated and crowd-based evaluation schemes. Our experiments show that the result of our crowd-based evaluation scheme is different from those of automated evaluation schemes. In addition, we regard the crowd-based evaluation results as ground truths and provide a quantitative performance measure to compare different automated evaluation schemes. We also discuss the impact of crowd workers on the results and show that the varying ability of crowd workers does not significantly impact the results.
△ Less
Submitted 30 August, 2021; v1 submitted 27 June, 2021;
originally announced July 2021.
-
Inter-domain Multi-relational Link Prediction
Authors:
Luu Huu Phuc,
Koh Takeuchi,
Seiji Okajima,
Arseny Tolmachev,
Tomoyoshi Takebayashi,
Koji Maruhashi,
Hisashi Kashima
Abstract:
Multi-relational graph is a ubiquitous and important data structure, allowing flexible representation of multiple types of interactions and relations between entities. Similar to other graph-structured data, link prediction is one of the most important tasks on multi-relational graphs and is often used for knowledge completion. When related graphs coexist, it is of great benefit to build a larger…
▽ More
Multi-relational graph is a ubiquitous and important data structure, allowing flexible representation of multiple types of interactions and relations between entities. Similar to other graph-structured data, link prediction is one of the most important tasks on multi-relational graphs and is often used for knowledge completion. When related graphs coexist, it is of great benefit to build a larger graph via integrating the smaller ones. The integration requires predicting hidden relational connections between entities belonged to different graphs (inter-domain link prediction). However, this poses a real challenge to existing methods that are exclusively designed for link prediction between entities of the same graph only (intra-domain link prediction). In this study, we propose a new approach to tackle the inter-domain link prediction problem by softly aligning the entity distributions between different domains with optimal transport and maximum mean discrepancy regularizers. Experiments on real-world datasets show that optimal transport regularizer is beneficial and considerably improves the performance of baseline methods.
△ Less
Submitted 9 July, 2021; v1 submitted 11 June, 2021;
originally announced June 2021.
-
A Multi-task Learning Framework for Gras**-Position Detection and Few-Shot Classification
Authors:
Yasuto Yokota,
Kanata Suzuki,
Yuzi Kanazawa,
Tomoyoshi Takebayashi
Abstract:
It is a big problem that a model of deep learning for a picking robot needs many labeled images. Operating costs of retraining a model becomes very expensive because the object shape of a product or a part often is changed in a factory. It is important to reduce the amount of labeled images required to train a model for a picking robot. In this study, we propose a multi-task learning framework for…
▽ More
It is a big problem that a model of deep learning for a picking robot needs many labeled images. Operating costs of retraining a model becomes very expensive because the object shape of a product or a part often is changed in a factory. It is important to reduce the amount of labeled images required to train a model for a picking robot. In this study, we propose a multi-task learning framework for few-shot classification using feature vectors from an intermediate layer of a model that detects gras** positions. In the field of manufacturing, multitask for shape classification and gras**-position detection is often required for picking robots. Prior multi-task learning studies include methods to learn one task with feature vectors from a deep neural network (DNN) learned for another task. However, the DNN that was used to detect gras** positions has two problems with respect to extracting feature vectors from a layer for shape classification: (1) Because each layer of the gras** position detection DNN is activated by all objects in the input image, it is necessary to refine the features for each gras** position. (2) It is necessary to select a layer to extract the features suitable for shape classification. To tackle these issues, we propose a method to refine the features for each gras** position and to select features from the optimal layer of the DNN. We then evaluated the shape classification accuracy using these features from the gras** positions. Our results confirm that our proposed framework can classify object shapes even when the input image includes multiple objects and the number of images available for training is small.
△ Less
Submitted 12 March, 2020;
originally announced March 2020.
-
Online Self-Supervised Learning for Object Picking: Detecting Optimum Gras** Position using a Metric Learning Approach
Authors:
Kanata Suzuki,
Yasuto Yokota,
Yuzi Kanazawa,
Tomoyoshi Takebayashi
Abstract:
Self-supervised learning methods are attractive candidates for automatic object picking. However, the trial samples lack the complete ground truth because the observable parts of the agent are limited. That is, the information contained in the trial samples is often insufficient to learn the specific gras** position of each object. Consequently, the training falls into a local solution, and the…
▽ More
Self-supervised learning methods are attractive candidates for automatic object picking. However, the trial samples lack the complete ground truth because the observable parts of the agent are limited. That is, the information contained in the trial samples is often insufficient to learn the specific gras** position of each object. Consequently, the training falls into a local solution, and the grasp positions learned by the robot are independent of the state of the object. In this study, the optimal gras** position of an individual object is determined from the gras** score, defined as the distance in the feature space obtained using metric learning. The closeness of the solution to the pre-designed optimal gras** position was evaluated in trials. The proposed method incorporates two types of feedback control: one feedback enlarges the gras** score when the gras** position approaches the optimum; the other reduces the negative feedback of the potential gras** positions among the gras** candidates. The proposed online self-supervised learning method employs two deep neural networks. : SSD that detects the gras** position of an object, and Siamese networks (SNs) that evaluate the trial sample using the similarity of two input data in the feature space. Our method embeds the relation of each gras** position as feature vectors by training the trial samples and a few pre-samples indicating the optimum gras** position. By incorporating the gras** score based on the feature space of SNs into the SSD training process, the method preferentially trains the optimum gras** position. In the experiment, the proposed method achieved a higher success rate than the baseline method using simple teaching signals. And the gras** scores in the feature space of the SNs accurately represented the gras** positions of the objects.
△ Less
Submitted 7 March, 2020;
originally announced March 2020.
