Showing 1–14 of 14 results for author: Nojima, Y

Integrating White and Black Box Techniques for Interpretable Machine Learning

Authors: Eric M. Vernon, Naoki Masuyama, Yusuke Nojima

Abstract: In machine learning algorithm design, there exists a trade-off between the interpretability and performance of the algorithm. In general, algorithms which are simpler and easier for humans to comprehend tend to show worse performance than more complex, less transparent algorithms. For example, a random forest classifier is likely to be more accurate than a simple decision tree, but at the expense… ▽ More In machine learning algorithm design, there exists a trade-off between the interpretability and performance of the algorithm. In general, algorithms which are simpler and easier for humans to comprehend tend to show worse performance than more complex, less transparent algorithms. For example, a random forest classifier is likely to be more accurate than a simple decision tree, but at the expense of interpretability. In this paper, we present an ensemble classifier design which classifies easier inputs using a highly-interpretable classifier (i.e., white box model), and more difficult inputs using a more powerful, but less interpretable classifier (i.e., black box model). △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: This paper was presented at ICICT2024 and will be published in conference proceedings by Springer LNNS. ISSN: 2367-3370, Series

Privacy-preserving Continual Federated Clustering via Adaptive Resonance Theory

Authors: Naoki Masuyama, Yusuke Nojima, Yuichiro Toda, Chu Kiong Loo, Hisao Ishibuchi, Naoyuki Kubota

Abstract: With the increasing importance of data privacy protection, various privacy-preserving machine learning methods have been proposed. In the clustering domain, various algorithms with a federated learning framework (i.e., federated clustering) have been actively studied and showed high clustering performance while preserving data privacy. However, most of the base clusterers (i.e., clustering algorit… ▽ More With the increasing importance of data privacy protection, various privacy-preserving machine learning methods have been proposed. In the clustering domain, various algorithms with a federated learning framework (i.e., federated clustering) have been actively studied and showed high clustering performance while preserving data privacy. However, most of the base clusterers (i.e., clustering algorithms) used in existing federated clustering algorithms need to specify the number of clusters in advance. These algorithms, therefore, are unable to deal with data whose distributions are unknown or continually changing. To tackle this problem, this paper proposes a privacy-preserving continual federated clustering algorithm. In the proposed algorithm, an adaptive resonance theory-based clustering algorithm capable of continual learning is used as a base clusterer. Therefore, the proposed algorithm inherits the ability of continual learning. Experimental results with synthetic and real-world datasets show that the proposed algorithm has superior clustering performance to state-of-the-art federated clustering algorithms while realizing data privacy protection and continual learning ability. The source code is available at \url{https://github.com/Masuyama-lab/FCAC}. △ Less

Submitted 7 September, 2023; originally announced September 2023.

Comments: This paper is currently under review. arXiv admin note: substantial text overlap with arXiv:2305.01507

A Parameter-free Adaptive Resonance Theory-based Topological Clustering Algorithm Capable of Continual Learning

Authors: Naoki Masuyama, Takanori Takebayashi, Yusuke Nojima, Chu Kiong Loo, Hisao Ishibuchi, Stefan Wermter

Abstract: In general, a similarity threshold (i.e., a vigilance parameter) for a node learning process in Adaptive Resonance Theory (ART)-based algorithms has a significant impact on clustering performance. In addition, an edge deletion threshold in a topological clustering algorithm plays an important role in adaptively generating well-separated clusters during a self-organizing process. In this paper, we… ▽ More In general, a similarity threshold (i.e., a vigilance parameter) for a node learning process in Adaptive Resonance Theory (ART)-based algorithms has a significant impact on clustering performance. In addition, an edge deletion threshold in a topological clustering algorithm plays an important role in adaptively generating well-separated clusters during a self-organizing process. In this paper, we propose a new parameter-free ART-based topological clustering algorithm capable of continual learning by introducing parameter estimation methods. Experimental results with synthetic and real-world datasets show that the proposed algorithm has superior clustering performance to the state-of-the-art clustering algorithms without any parameter pre-specifications. △ Less

Submitted 2 May, 2023; v1 submitted 30 April, 2023; originally announced May 2023.

Comments: This paper is currently under review

Direct Observation and Quantitative Measurement of OH Radical Desorption During the Ultraviolet Photolysis of Liquid Nonanoic Acid

Authors: Naoki Numadate, Shota Saito, Yuki Nojima, Taka-aki Ishibashi, Shinichi Enami, Tetsuya Hama

Abstract: Ultraviolet (UV) photolysis of fatty acid surfactants which cover the surfaces of atmospheric liquid aerosols and are found in the oceans such as nonanoic acid (NA) has recently been suggested as a source of hydroxyl (OH) radicals in the troposphere. We used laser-induced fluorescence to directly observe OH radicals desorbed from the surface of neat liquid NA as a primary photoproduct following 21… ▽ More Ultraviolet (UV) photolysis of fatty acid surfactants which cover the surfaces of atmospheric liquid aerosols and are found in the oceans such as nonanoic acid (NA) has recently been suggested as a source of hydroxyl (OH) radicals in the troposphere. We used laser-induced fluorescence to directly observe OH radicals desorbed from the surface of neat liquid NA as a primary photoproduct following 213 nm irradiation. The upper limit of photoreaction cross section for the OH radical desorption was estimated to be 9.0(4.1) $\times$ 10$^{-22}$ cm$^2$, which is only 1.2$\pm$0.8% of the photoreaction cross section established for the photolysis of gas-phase acetic acid monomers. Vibrational sum-frequency generation spectroscopy for liquid NA revealed the hydrogen-bonded, cyclic, dimer structure of the NA molecules at the liquid surface. This dimerization can inhibit the formation of OH radicals and lead the present low photochemical reactivity of liquid NA. △ Less

Submitted 13 September, 2022; originally announced September 2022.

Reference Vector Adaptation and Mating Selection Strategy via Adaptive Resonance Theory-based Clustering for Many-objective Optimization

Authors: Takato Kinoshita, Naoki Masuyama, Yi** Liu, Yusuke Nojima, Hisao Ishibuchi

Abstract: Decomposition-based multiobjective evolutionary algorithms (MOEAs) with clustering-based reference vector adaptation show good optimization performance for many-objective optimization problems (MaOPs). Especially, algorithms that employ a clustering algorithm with a topological structure (i.e., a network composed of nodes and edges) show superior optimization performance to other MOEAs for MaOPs w… ▽ More Decomposition-based multiobjective evolutionary algorithms (MOEAs) with clustering-based reference vector adaptation show good optimization performance for many-objective optimization problems (MaOPs). Especially, algorithms that employ a clustering algorithm with a topological structure (i.e., a network composed of nodes and edges) show superior optimization performance to other MOEAs for MaOPs with irregular Pareto optimal fronts (PFs). These algorithms, however, do not effectively utilize information of the topological structure in the search process. Moreover, the clustering algorithms typically used in conventional studies have limited clustering performance, inhibiting the ability to extract useful information for the search process. This paper proposes an adaptive reference vector-guided evolutionary algorithm using an adaptive resonance theory-based clustering with a topological structure. The proposed algorithm utilizes the information of the topological structure not only for reference vector adaptation but also for mating selection. The proposed algorithm is compared with 8 state-of-the-art MOEAs on 78 test problems. Experimental results reveal the outstanding optimization performance of the proposed algorithm over the others on MaOPs with various properties. △ Less

Submitted 3 May, 2022; v1 submitted 22 April, 2022; originally announced April 2022.

Comments: This paper is currently under review

Class-wise Classifier Design Capable of Continual Learning using Adaptive Resonance Theory-based Topological Clustering

Authors: Naoki Masuyama, Yusuke Nojima, Farhan Dawood, Zongying Liu

Abstract: This paper proposes a supervised classification algorithm capable of continual learning by utilizing an Adaptive Resonance Theory (ART)-based growing self-organizing clustering algorithm. The ART-based clustering algorithm is theoretically capable of continual learning, and the proposed algorithm independently applies it to each class of training data for generating classifiers. Whenever an additi… ▽ More This paper proposes a supervised classification algorithm capable of continual learning by utilizing an Adaptive Resonance Theory (ART)-based growing self-organizing clustering algorithm. The ART-based clustering algorithm is theoretically capable of continual learning, and the proposed algorithm independently applies it to each class of training data for generating classifiers. Whenever an additional training data set from a new class is given, a new ART-based clustering will be defined in a different learning space. Thanks to the above-mentioned features, the proposed algorithm realizes continual learning capability. Simulation experiments showed that the proposed algorithm has superior classification performance compared with state-of-the-art clustering-based classification algorithms capable of continual learning. △ Less

Submitted 21 September, 2023; v1 submitted 18 March, 2022; originally announced March 2022.

Comments: This paper is currently under review. arXiv admin note: substantial text overlap with arXiv:2201.10713

Journal ref: Applied Sciences, vol. 13, no. 21, #11980, 2023

Adaptive Resonance Theory-based Topological Clustering with a Divisive Hierarchical Structure Capable of Continual Learning

Authors: Naoki Masuyama, Narito Amako, Yuna Yamada, Yusuke Nojima, Hisao Ishibuchi

Abstract: Adaptive Resonance Theory (ART) is considered as an effective approach for realizing continual learning thanks to its ability to handle the plasticity-stability dilemma. In general, however, the clustering performance of ART-based algorithms strongly depends on the specification of a similarity threshold, i.e., a vigilance parameter, which is data-dependent and specified by hand. This paper propos… ▽ More Adaptive Resonance Theory (ART) is considered as an effective approach for realizing continual learning thanks to its ability to handle the plasticity-stability dilemma. In general, however, the clustering performance of ART-based algorithms strongly depends on the specification of a similarity threshold, i.e., a vigilance parameter, which is data-dependent and specified by hand. This paper proposes an ART-based topological clustering algorithm with a mechanism that automatically estimates a similarity threshold from the distribution of data points. In addition, for improving information extraction performance, a divisive hierarchical clustering algorithm capable of continual learning is proposed by introducing a hierarchical structure to the proposed algorithm. Experimental results demonstrate that the proposed algorithm has high clustering performance comparable with recently-proposed state-of-the-art hierarchical clustering algorithms. △ Less

Submitted 7 July, 2022; v1 submitted 25 January, 2022; originally announced January 2022.

Comments: This paper is accepted in IEEE Access

Journal ref: IEEE Access, vol. 10, pp. 68042-68056, June 2022

AI-Fuzzy Markup Language with Computational Intelligence for High-School Student Learning

Authors: Chang-Shing Lee, Mei-Hui Wang, Yusuke Nojima, Marek Reformat, Leo Guo

Abstract: Computational Intelligence (CI), which includes fuzzy logic (FL), neural network (NN), and evolutionary computation (EC), is an imperative branch of artificial intelligence (AI). As a core technology of AI, it plays a vital role in develo** intelligent systems, such as games and game engines, neural-based systems including a variety of deep network models, evolutionary-based optimization methods… ▽ More Computational Intelligence (CI), which includes fuzzy logic (FL), neural network (NN), and evolutionary computation (EC), is an imperative branch of artificial intelligence (AI). As a core technology of AI, it plays a vital role in develo** intelligent systems, such as games and game engines, neural-based systems including a variety of deep network models, evolutionary-based optimization methods, and advanced cognitive techniques. The 2021 IEEE CIS Summer School on CI for High-School Student Learning was held physically at the JanFuSun Resort Hotel, Taiwan, and virtually on Zoom, on August 10-12, 2021. The main contents of the Summer School were lectures focused on the basics of FL, NN, and EC and the workshop on AIoT (Artificial Intelligence of Things). Invited speakers gave nine courses covering topics like CI real-world applications, fundamentals of FL, and the introduction to NN and EC. The 2021 Summer School was supported by the 2021 IEEE CIS High School Outreach Subcommittee. We also invited students and teachers of high and elementary schools from Taiwan, Japan, and Indonesia. They attended the school and participated in AIoT workshop, gaining experience in applications of AIoT-FML learning tools. According to the short report and feedback from the involved students and teachers, we find out that most participants have quickly understood the principles of CI, FL, NN, and EC. In addition, one of the teachers sent the following remark to the organizers: "This is a great event to introduce students to computational intelligence at a young age, stimulate them to be involved in rapidly evolving fields, and foster participation in future research adventures." △ Less

Submitted 6 November, 2021; originally announced December 2021.

Comments: 4 pages, 7 figures, and 1 table

Effects of Different Optimization Formulations in Evolutionary Reinforcement Learning on Diverse Behavior Generation

Authors: Victor Villin, Naoki Masuyama, Yusuke Nojima

Abstract: Generating various strategies for a given task is challenging. However, it has already proven to bring many assets to the main learning process, such as improved behavior exploration. With the growth in the interest of heterogeneity in solution in evolutionary computation and reinforcement learning, many promising approaches have emerged. To better understand how one guides multiple policies towar… ▽ More Generating various strategies for a given task is challenging. However, it has already proven to bring many assets to the main learning process, such as improved behavior exploration. With the growth in the interest of heterogeneity in solution in evolutionary computation and reinforcement learning, many promising approaches have emerged. To better understand how one guides multiple policies toward distinct strategies and benefit from diversity, we need to analyze further the influence of the reward signal modulation and other evolutionary mechanisms on the obtained behaviors. To that effect, this paper considers an existing evolutionary reinforcement learning framework which exploits multi-objective optimization as a way to obtain policies that succeed at behavior-related tasks as well as completing the main goal. Experiments on the Atari games stress that optimization formulations which do not consider objectives equally fail at generating diversity and even output agents that are worse at solving the problem at hand, regardless of the obtained behaviors. △ Less

Submitted 27 January, 2022; v1 submitted 15 October, 2021; originally announced October 2021.

Journal ref: V. Villin, N. Masuyama and Y. Nojima, "Effects of Different Optimization Formulations in Evolutionary Reinforcement Learning on Diverse Behavior Generation," 2021 IEEE Symposium Series on Computational Intelligence (SSCI), 2021, pp. 01-08

Multi-label Classification via Adaptive Resonance Theory-based Clustering

Authors: Naoki Masuyama, Yusuke Nojima, Chu Kiong Loo, Hisao Ishibuchi

Abstract: This paper proposes a multi-label classification algorithm capable of continual learning by applying an Adaptive Resonance Theory (ART)-based clustering algorithm and the Bayesian approach for label probability computation. The ART-based clustering algorithm adaptively and continually generates prototype nodes corresponding to given data, and the generated nodes are used as classifiers. The label… ▽ More This paper proposes a multi-label classification algorithm capable of continual learning by applying an Adaptive Resonance Theory (ART)-based clustering algorithm and the Bayesian approach for label probability computation. The ART-based clustering algorithm adaptively and continually generates prototype nodes corresponding to given data, and the generated nodes are used as classifiers. The label probability computation independently counts the number of label appearances for each class and calculates the Bayesian probabilities. Thus, the label probability computation can cope with an increase in the number of labels. Experimental results with synthetic and real-world multi-label datasets show that the proposed algorithm has competitive classification performance to other well-known algorithms while realizing continual learning. △ Less

Submitted 3 September, 2021; v1 submitted 2 March, 2021; originally announced March 2021.

Identifying Properties of Real-World Optimisation Problems through a Questionnaire

Authors: Koen van der Blom, Timo M. Deist, Vanessa Volz, Mariapia Marchi, Yusuke Nojima, Boris Naujoks, Akira Oyama, Tea Tušar

Abstract: Optimisation algorithms are commonly compared on benchmarks to get insight into performance differences. However, it is not clear how closely benchmarks match the properties of real-world problems because these properties are largely unknown. This work investigates the properties of real-world problems through a questionnaire to enable the design of future benchmark problems that more closely rese… ▽ More Optimisation algorithms are commonly compared on benchmarks to get insight into performance differences. However, it is not clear how closely benchmarks match the properties of real-world problems because these properties are largely unknown. This work investigates the properties of real-world problems through a questionnaire to enable the design of future benchmark problems that more closely resemble those found in the real world. The results, while not representative as they are based on only 45 responses, indicate that many problems possess at least one of the following properties: they are constrained, deterministic, have only continuous variables, require substantial computation times for both the objectives and the constraints, or allow a limited number of evaluations. Properties like known optimal solutions and analytical gradients are rarely available, limiting the options in guiding the optimisation process. These are all important aspects to consider when designing realistic benchmark problems. At the same time, the design of realistic benchmarks is difficult, because objective functions are often reported to be black-box and many problem properties are unknown. To further improve the understanding of real-world problems, readers working on a real-world optimisation problem are encouraged to fill out the questionnaire: https://tinyurl.com/opt-survey △ Less

Submitted 14 July, 2021; v1 submitted 11 November, 2020; originally announced November 2020.

Comments: Book Chapter (Under review, revised version)

Towards Realistic Optimization Benchmarks: A Questionnaire on the Properties of Real-World Problems

Authors: Koen van der Blom, Timo M. Deist, Tea Tušar, Mariapia Marchi, Yusuke Nojima, Akira Oyama, Vanessa Volz, Boris Naujoks

Abstract: Benchmarks are a useful tool for empirical performance comparisons. However, one of the main shortcomings of existing benchmarks is that it remains largely unclear how they relate to real-world problems. What does an algorithm's performance on a benchmark say about its potential on a specific real-world problem? This work aims to identify properties of real-world problems through a questionnaire o… ▽ More Benchmarks are a useful tool for empirical performance comparisons. However, one of the main shortcomings of existing benchmarks is that it remains largely unclear how they relate to real-world problems. What does an algorithm's performance on a benchmark say about its potential on a specific real-world problem? This work aims to identify properties of real-world problems through a questionnaire on real-world single-, multi-, and many-objective optimization problems. Based on initial responses, a few challenges that have to be considered in the design of realistic benchmarks can already be identified. A key point for future work is to gather more responses to the questionnaire to allow an analysis of common combinations of properties. In turn, such common combinations can then be included in improved benchmark suites. To gather more data, the reader is invited to participate in the questionnaire at: https://tinyurl.com/opt-survey △ Less

Submitted 14 April, 2020; originally announced April 2020.

Comments: 2 pages, GECCO2020 Poster Paper

A GFML-based Robot Agent for Human and Machine Cooperative Learning on Game of Go

Authors: Chang-Shing Lee, Mei-Hui Wang, Li-Chuang Chen, Yusuke Nojima, Tzong-Xiang Huang, **seok Woo, Naoyuki Kubota, Eri Sato-Shimokawara, Toru Yamaguchi

Abstract: This paper applies a genetic algorithm and fuzzy markup language to construct a human and smart machine cooperative learning system on game of Go. The genetic fuzzy markup language (GFML)-based Robot Agent can work on various kinds of robots, including Palro, Pepper, and TMUs robots. We use the parameters of FAIR open source Darkforest and OpenGo AI bots to construct the knowledge base of Open Go… ▽ More This paper applies a genetic algorithm and fuzzy markup language to construct a human and smart machine cooperative learning system on game of Go. The genetic fuzzy markup language (GFML)-based Robot Agent can work on various kinds of robots, including Palro, Pepper, and TMUs robots. We use the parameters of FAIR open source Darkforest and OpenGo AI bots to construct the knowledge base of Open Go Darkforest (OGD) cloud platform for student learning on the Internet. In addition, we adopt the data from AlphaGo Master sixty online games as the training data to construct the knowledge base and rule base of the co-learning system. First, the Darkforest predicts the win rate based on various simulation numbers and matching rates for each game on OGD platform, then the win rate of OpenGo is as the final desired output. The experimental results show that the proposed approach can improve knowledge base and rule base of the prediction ability based on Darkforest and OpenGo AI bot with various simulation numbers. △ Less

Submitted 22 January, 2019; originally announced January 2019.

FML-based Prediction Agent and Its Application to Game of Go

Authors: Chang-Shing Lee, Mei-Hui Wang, Chia-Hsiu Kao, Sheng-Chi Yang, Yusuke Nojima, Ryosuke Saga, Nan Shuo, Naoyuki Kubota

Abstract: In this paper, we present a robotic prediction agent including a darkforest Go engine, a fuzzy markup language (FML) assessment engine, an FML-based decision support engine, and a robot engine for game of Go application. The knowledge base and rule base of FML assessment engine are constructed by referring the information from the darkforest Go engine located in NUTN and OPU, for example, the numb… ▽ More In this paper, we present a robotic prediction agent including a darkforest Go engine, a fuzzy markup language (FML) assessment engine, an FML-based decision support engine, and a robot engine for game of Go application. The knowledge base and rule base of FML assessment engine are constructed by referring the information from the darkforest Go engine located in NUTN and OPU, for example, the number of MCTS simulations and winning rate prediction. The proposed robotic prediction agent first retrieves the database of Go competition website, and then the FML assessment engine infers the winning possibility based on the information generated by darkforest Go engine. The FML-based decision support engine computes the winning possibility based on the partial game situation inferred by FML assessment engine. Finally, the robot engine combines with the human-friendly robot partner PALRO, produced by Fujisoft incorporated, to report the game situation to human Go players. Experimental results show that the FML-based prediction agent can work effectively. △ Less

Submitted 16 April, 2017; originally announced April 2017.

Comments: 6 pages, 12 figures, Joint 17th World Congress of International Fuzzy Systems Association and 9th International Conference on Soft Computing and Intelligent Systems (IFSA-SCIS 2017), Otsu, Japan, Jun. 27-30, 2017