-
Improving Performance Prediction of Electrolyte Formulations with Transformer-based Molecular Representation Model
Authors:
Indra Priyadarsini,
Vidushi Sharma,
Seiji Takeda,
Akihiro Kishimoto,
Lisa Hamada,
Hajime Shinohara
Abstract:
Development of efficient and high-performing electrolytes is crucial for advancing energy storage technologies, particularly in batteries. Predicting the performance of battery electrolytes rely on complex interactions between the individual constituents. Consequently, a strategy that adeptly captures these relationships and forms a robust representation of the formulation is essential for integra…
▽ More
Development of efficient and high-performing electrolytes is crucial for advancing energy storage technologies, particularly in batteries. Predicting the performance of battery electrolytes rely on complex interactions between the individual constituents. Consequently, a strategy that adeptly captures these relationships and forms a robust representation of the formulation is essential for integrating with machine learning models to predict properties accurately. In this paper, we introduce a novel approach leveraging a transformer-based molecular representation model to effectively and efficiently capture the representation of electrolyte formulations. The performance of the proposed approach is evaluated on two battery property prediction tasks and the results show superior performance compared to the state-of-the-art methods.
△ Less
Submitted 28 June, 2024;
originally announced June 2024.
-
Optimal Transport with Cyclic Symmetry
Authors:
Shoichiro Takeda,
Yasunori Akagi,
Naoki Marumo,
Kenta Niwa
Abstract:
We propose novel fast algorithms for optimal transport (OT) utilizing a cyclic symmetry structure of input data. Such OT with cyclic symmetry appears universally in various real-world examples: image processing, urban planning, and graph processing. Our main idea is to reduce OT to a small optimization problem that has significantly fewer variables by utilizing cyclic symmetry and various optimiza…
▽ More
We propose novel fast algorithms for optimal transport (OT) utilizing a cyclic symmetry structure of input data. Such OT with cyclic symmetry appears universally in various real-world examples: image processing, urban planning, and graph processing. Our main idea is to reduce OT to a small optimization problem that has significantly fewer variables by utilizing cyclic symmetry and various optimization techniques. On the basis of this reduction, our algorithms solve the small optimization problem instead of the original OT. As a result, our algorithms obtain the optimal solution and the objective function value of the original OT faster than solving the original OT directly. In this paper, our focus is on two crucial OT formulations: the linear programming OT (LOT) and the strongly convex-regularized OT, which includes the well-known entropy-regularized OT (EROT). Experiments show the effectiveness of our algorithms for LOT and EROT in synthetic/real-world data that has a strict/approximate cyclic symmetry structure. Through theoretical and experimental results, this paper successfully introduces the concept of symmetry into the OT research field for the first time.
△ Less
Submitted 21 November, 2023;
originally announced November 2023.
-
Improving Molecular Properties Prediction Through Latent Space Fusion
Authors:
Eduardo Soares,
Akihiro Kishimoto,
Emilio Vital Brazil,
Seiji Takeda,
Hiroshi Ka**o,
Renato Cerqueira
Abstract:
Pre-trained Language Models have emerged as promising tools for predicting molecular properties, yet their development is in its early stages, necessitating further research to enhance their efficacy and address challenges such as generalization and sample efficiency. In this paper, we present a multi-view approach that combines latent spaces derived from state-of-the-art chemical models. Our appr…
▽ More
Pre-trained Language Models have emerged as promising tools for predicting molecular properties, yet their development is in its early stages, necessitating further research to enhance their efficacy and address challenges such as generalization and sample efficiency. In this paper, we present a multi-view approach that combines latent spaces derived from state-of-the-art chemical models. Our approach relies on two pivotal elements: the embeddings derived from MHG-GNN, which represent molecular structures as graphs, and MoLFormer embeddings rooted in chemical language. The attention mechanism of MoLFormer is able to identify relations between two atoms even when their distance is far apart, while the GNN of MHG-GNN can more precisely capture relations among multiple atoms closely located. In this work, we demonstrate the superior performance of our proposed multi-view approach compared to existing state-of-the-art methods, including MoLFormer-XL, which was trained on 1.1 billion molecules, particularly in intricate tasks such as predicting clinical trial drug toxicity and inhibiting HIV replication. We assessed our approach using six benchmark datasets from MoleculeNet, where it outperformed competitors in five of them. Our study highlights the potential of latent space fusion and feature integration for advancing molecular property prediction. In this work, we use small versions of MHG-GNN and MoLFormer, which opens up an opportunity for further improvement when our approach uses a larger-scale dataset.
△ Less
Submitted 20 October, 2023;
originally announced October 2023.
-
MHG-GNN: Combination of Molecular Hypergraph Grammar with Graph Neural Network
Authors:
Akihiro Kishimoto,
Hiroshi Ka**o,
Masataka Hirose,
Junta Fuchiwaki,
Indra Priyadarsini,
Lisa Hamada,
Hajime Shinohara,
Daiju Nakano,
Seiji Takeda
Abstract:
Property prediction plays an important role in material discovery. As an initial step to eventually develop a foundation model for material science, we introduce a new autoencoder called the MHG-GNN, which combines graph neural network (GNN) with Molecular Hypergraph Grammar (MHG). Results on a variety of property prediction tasks with diverse materials show that MHG-GNN is promising.
Property prediction plays an important role in material discovery. As an initial step to eventually develop a foundation model for material science, we introduce a new autoencoder called the MHG-GNN, which combines graph neural network (GNN) with Molecular Hypergraph Grammar (MHG). Results on a variety of property prediction tasks with diverse materials show that MHG-GNN is promising.
△ Less
Submitted 28 September, 2023;
originally announced September 2023.
-
Adversarial Finetuning with Latent Representation Constraint to Mitigate Accuracy-Robustness Tradeoff
Authors:
Satoshi Suzuki,
Shin'ya Yamaguchi,
Shoichiro Takeda,
Sekitoshi Kanai,
Naoki Makishima,
Atsushi Ando,
Ryo Masumura
Abstract:
This paper addresses the tradeoff between standard accuracy on clean examples and robustness against adversarial examples in deep neural networks (DNNs). Although adversarial training (AT) improves robustness, it degrades the standard accuracy, thus yielding the tradeoff. To mitigate this tradeoff, we propose a novel AT method called ARREST, which comprises three components: (i) adversarial finetu…
▽ More
This paper addresses the tradeoff between standard accuracy on clean examples and robustness against adversarial examples in deep neural networks (DNNs). Although adversarial training (AT) improves robustness, it degrades the standard accuracy, thus yielding the tradeoff. To mitigate this tradeoff, we propose a novel AT method called ARREST, which comprises three components: (i) adversarial finetuning (AFT), (ii) representation-guided knowledge distillation (RGKD), and (iii) noisy replay (NR). AFT trains a DNN on adversarial examples by initializing its parameters with a DNN that is standardly pretrained on clean examples. RGKD and NR respectively entail a regularization term and an algorithm to preserve latent representations of clean examples during AFT. RGKD penalizes the distance between the representations of the standardly pretrained and AFT DNNs. NR switches input adversarial examples to nonadversarial ones when the representation changes significantly during AFT. By combining these components, ARREST achieves both high standard accuracy and robustness. Experimental results demonstrate that ARREST mitigates the tradeoff more effectively than previous AT-based methods do.
△ Less
Submitted 31 August, 2023;
originally announced August 2023.
-
Accelerating Material Design with the Generative Toolkit for Scientific Discovery
Authors:
Matteo Manica,
Jannis Born,
Joris Cadow,
Dimitrios Christofidellis,
Ashish Dave,
Dean Clarke,
Yves Gaetan Nana Teukam,
Giorgio Giannone,
Samuel C. Hoffman,
Matthew Buchan,
Vijil Chenthamarakshan,
Timothy Donovan,
Hsiang Han Hsu,
Federico Zipoli,
Oliver Schilter,
Akihiro Kishimoto,
Lisa Hamada,
Inkit Padhi,
Karl Wehden,
Lauren McHugh,
Alexy Khrabrov,
Payel Das,
Seiji Takeda,
John R. Smith
Abstract:
With the growing availability of data within various scientific domains, generative models hold enormous potential to accelerate scientific discovery. They harness powerful representations learned from datasets to speed up the formulation of novel hypotheses with the potential to impact material discovery broadly. We present the Generative Toolkit for Scientific Discovery (GT4SD). This extensible…
▽ More
With the growing availability of data within various scientific domains, generative models hold enormous potential to accelerate scientific discovery. They harness powerful representations learned from datasets to speed up the formulation of novel hypotheses with the potential to impact material discovery broadly. We present the Generative Toolkit for Scientific Discovery (GT4SD). This extensible open-source library enables scientists, developers, and researchers to train and use state-of-the-art generative models to accelerate scientific discovery focused on material design.
△ Less
Submitted 31 January, 2023; v1 submitted 8 July, 2022;
originally announced July 2022.
-
Molecule Generation Experience: An Open Platform of Material Design for Public Users
Authors:
Seiji Takeda,
Toshiyuki Hama,
Hsiang-Han Hsu,
Akihiro Kishimoto,
Makoto Kogoh,
Takumi Hongo,
Kumiko Fujieda,
Hideaki Nakashika,
Dmitry Zubarev,
Daniel P. Sanders,
Jed W. Pitera,
Junta Fuchiwaki,
Daiju Nakano
Abstract:
Artificial Intelligence (AI)-driven material design has been attracting great attentions as a groundbreaking technology across a wide spectrum of industries. Molecular design is particularly important owing to its broad application domains and boundless creativity attributed to progresses in generative models. The recent maturity of molecular generative models has stimulated expectations for pract…
▽ More
Artificial Intelligence (AI)-driven material design has been attracting great attentions as a groundbreaking technology across a wide spectrum of industries. Molecular design is particularly important owing to its broad application domains and boundless creativity attributed to progresses in generative models. The recent maturity of molecular generative models has stimulated expectations for practical use among potential users, who are not necessarily familiar with coding or scripting, such as experimental engineers and students in chemical domains. However, most of the existing molecular generative models are Python libraries on GitHub, that are accessible for only IT-savvy users. To fill this gap, we newly developed a graphical user interface (GUI)-based web application of molecular generative models, Molecule Generation Experience, that is open to the general public. This is the first web application of molecular generative models enabling users to work with built-in datasets to carry out molecular design. In this paper, we describe the background technology extended from our previous work. Our new online evaluation and structural filtering algorithms significantly improved the generation speed by 30 to 1,000 times with a wider structural variety, satisfying chemical stability and synthetic reality. We also describe in detail our Kubernetes-based scalable cloud architecture and user-oriented GUI that are necessary components to achieve a public service. Finally, we present actual use cases in industrial research to design new photoacid generators (PAGs) as well as release cases in educational events.
△ Less
Submitted 6 August, 2021;
originally announced August 2021.
-
A dataset for complex activity recognition withmicro and macro activities in a cooking scenario
Authors:
Paula Lago,
Shingo Takeda,
Sayeda Shamma Alia,
Kohei Adachi,
Brahim Bennai,
Francois Charpillet,
Sozo Inoue
Abstract:
Complex activity recognition can benefit from understanding the steps that compose them. Current datasets, however, are annotated with one label only, hindering research in this direction. In this paper, we describe a new dataset for sensor-based activity recognition featuring macro and micro activities in a cooking scenario. Three sensing systems measured simultaneously, namely a motion capture s…
▽ More
Complex activity recognition can benefit from understanding the steps that compose them. Current datasets, however, are annotated with one label only, hindering research in this direction. In this paper, we describe a new dataset for sensor-based activity recognition featuring macro and micro activities in a cooking scenario. Three sensing systems measured simultaneously, namely a motion capture system, tracking 25 points on the body; two smartphone accelerometers, one on the hip and the other one on the forearm; and two smartwatches one on each wrist. The dataset is labeled for both the recipes (macro activities) and the steps (micro activities). We summarize the results of a baseline classification using traditional activity recognition pipelines. The dataset is designed to be easily used to test and develop activity recognition approaches.
△ Less
Submitted 18 June, 2020;
originally announced June 2020.
-
Molecular Inverse-Design Platform for Material Industries
Authors:
Seiji Takeda,
Toshiyuki Hama,
Hsiang-Han Hsu,
Victoria A. Piunova,
Dmitry Zubarev,
Daniel P. Sanders,
Jed W. Pitera,
Makoto Kogoh,
Takumi Hongo,
Yenwei Cheng,
Wolf Bocanett,
Hideaki Nakashika,
Akihiro Fujita,
Yuta Tsuchiya,
Katsuhiko Hino,
Kentaro Yano,
Shuichi Hirose,
Hiroki Toda,
Yasumitsu Orii,
Daiju Nakano
Abstract:
The discovery of new materials has been the essential force which brings a discontinuous improvement to industrial products' performance. However, the extra-vast combinatorial design space of material structures exceeds human experts' capability to explore all, thereby hampering material development. In this paper, we present a material industry-oriented web platform of an AI-driven molecular inve…
▽ More
The discovery of new materials has been the essential force which brings a discontinuous improvement to industrial products' performance. However, the extra-vast combinatorial design space of material structures exceeds human experts' capability to explore all, thereby hampering material development. In this paper, we present a material industry-oriented web platform of an AI-driven molecular inverse-design system, which automatically designs brand new molecular structures rapidly and diversely. Different from existing inverse-design solutions, in this system, the combination of substructure-based feature encoding and molecular graph generation algorithms allows a user to gain high-speed, interpretable, and customizable design process. Also, a hierarchical data structure and user-oriented UI provide a flexible and intuitive workflow. The system is deployed on IBM's and our client's cloud servers and has been used by 5 partner companies. To illustrate actual industrial use cases, we exhibit inverse-design of sugar and dye molecules, that were carried out by experimental chemists in those client companies. Compared to general human chemist's standard performance, the molecular design speed was accelerated more than 10 times, and greatly increased variety was observed in the inverse-designed molecules without loss of chemical realism.
△ Less
Submitted 16 May, 2020; v1 submitted 23 April, 2020;
originally announced April 2020.
-
AI-driven Inverse Design System for Organic Molecules
Authors:
Seiji Takeda,
Toshiyuki Hama,
Hsiang-Han Hsu,
Toshiyuki Yamane,
Koji Masuda,
Victoria A. Piunova,
Dmitry Zubarev,
Jed Pitera,
Daniel P. Sanders,
Daiju Nakano
Abstract:
Designing novel materials that possess desired properties is a central need across many manufacturing industries. Driven by that industrial need, a variety of algorithms and tools have been developed that combine AI (machine learning and analytics) with domain knowledge in physics, chemistry, and materials science. AI-driven materials design can be divided to mainly two stages; the first one is th…
▽ More
Designing novel materials that possess desired properties is a central need across many manufacturing industries. Driven by that industrial need, a variety of algorithms and tools have been developed that combine AI (machine learning and analytics) with domain knowledge in physics, chemistry, and materials science. AI-driven materials design can be divided to mainly two stages; the first one is the modeling stage, where the goal is to build an accurate regression or classification model to predict material properties (e.g. glass transition temperature) or attributes (e.g. toxic/non-toxic). The next stage is design, where the goal is to assemble or tune material structures so that they can achieve user-demanded target property values based on a prediction model that is trained in the modeling stage. For maximum benefit, these two stages should be architected to form a coherent workflow. Today there are several emerging services and tools for AI-driven material design, however, most of them provide only partial technical components (e.g. data analyzer, regression model, structure generator, etc.), that are useful for specific purposes, but for comprehensive material design, those components need to be orchestrated appropriately. Our material design system provides an end-to-end solution to this problem, with a workflow that consists of data input, feature encoding, prediction modeling, solution search, and structure generation. The system builds a regression model to predict properties, solves an inverse problem on the trained model, and generates novel chemical structure candidates that satisfy the target properties. In this paper we will introduce the methodology of our system, and demonstrate a simple example of inverse design generating new chemical structures that satisfy targeted physical property values.
△ Less
Submitted 20 January, 2020;
originally announced January 2020.
-
Image Enhanced Rotation Prediction for Self-Supervised Learning
Authors:
Shin'ya Yamaguchi,
Sekitoshi Kanai,
Tetsuya Shioda,
Shoichiro Takeda
Abstract:
The rotation prediction (Rotation) is a simple pretext-task for self-supervised learning (SSL), where models learn useful representations for target vision tasks by solving pretext-tasks. Although Rotation captures information of object shapes, it hardly captures information of textures. To tackle this problem, we introduce a novel pretext-task called image enhanced rotation prediction (IE-Rot) fo…
▽ More
The rotation prediction (Rotation) is a simple pretext-task for self-supervised learning (SSL), where models learn useful representations for target vision tasks by solving pretext-tasks. Although Rotation captures information of object shapes, it hardly captures information of textures. To tackle this problem, we introduce a novel pretext-task called image enhanced rotation prediction (IE-Rot) for SSL. IE-Rot simultaneously solves Rotation and another pretext-task based on image enhancement (e.g., sharpening and solarizing) while maintaining simplicity. Through the simultaneous prediction of rotation and image enhancement, models learn representations to capture the information of not only object shapes but also textures. Our experimental results show that IE-Rot models outperform Rotation on various standard benchmarks including ImageNet classification, PASCAL-VOC detection, and COCO detection/segmentation.
△ Less
Submitted 4 June, 2021; v1 submitted 25 December, 2019;
originally announced December 2019.
-
Recent Advances in Physical Reservoir Computing: A Review
Authors:
Gouhei Tanaka,
Toshiyuki Yamane,
Jean Benoit Héroux,
Ryosho Nakane,
Naoki Kanazawa,
Seiji Takeda,
Hidetoshi Numata,
Daiju Nakano,
Akira Hirose
Abstract:
Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for map** inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir…
▽ More
Reservoir computing is a computational framework suited for temporal/sequential data processing. It is derived from several recurrent neural network models, including echo state networks and liquid state machines. A reservoir computing system consists of a reservoir for map** inputs into a high-dimensional space and a readout for pattern analysis from the high-dimensional states in the reservoir. The reservoir is fixed and only the readout is trained with a simple method such as linear regression and classification. Thus, the major advantage of reservoir computing compared to other recurrent neural networks is fast learning, resulting in low training cost. Another advantage is that the reservoir without adaptive updating is amenable to hardware implementation using a variety of physical systems, substrates, and devices. In fact, such physical reservoir computing has attracted increasing attention in diverse fields of research. The purpose of this review is to provide an overview of recent advances in physical reservoir computing by classifying them according to the type of the reservoir. We discuss the current issues and perspectives related to physical reservoir computing, in order to further expand its practical applications and develop next-generation machine learning systems.
△ Less
Submitted 15 April, 2019; v1 submitted 15 August, 2018;
originally announced August 2018.
