-
Understanding Diffusion Models by Feynman's Path Integral
Authors:
Yuji Hirono,
Akinori Tanaka,
Kenji Fukushima
Abstract:
Score-based diffusion models have proven effective in image generation and have gained widespread usage; however, the underlying factors contributing to the performance disparity between stochastic and deterministic (i.e., the probability flow ODEs) sampling schemes remain unclear. We introduce a novel formulation of diffusion models using Feynman's path integral, which is a formulation originally…
▽ More
Score-based diffusion models have proven effective in image generation and have gained widespread usage; however, the underlying factors contributing to the performance disparity between stochastic and deterministic (i.e., the probability flow ODEs) sampling schemes remain unclear. We introduce a novel formulation of diffusion models using Feynman's path integral, which is a formulation originally developed for quantum physics. We find this formulation providing comprehensive descriptions of score-based generative models, and demonstrate the derivation of backward stochastic differential equations and loss functions.The formulation accommodates an interpolating parameter connecting stochastic and deterministic sampling schemes, and we identify this parameter as a counterpart of Planck's constant in quantum physics. This analogy enables us to apply the Wentzel-Kramers-Brillouin (WKB) expansion, a well-established technique in quantum physics, for evaluating the negative log-likelihood to assess the performance disparity between stochastic and deterministic sampling schemes.
△ Less
Submitted 17 March, 2024;
originally announced March 2024.
-
EdgePruner: Poisoned Edge Pruning in Graph Contrastive Learning
Authors:
Hiroya Kato,
Kento Hasegawa,
Seira Hidano,
Kazuhide Fukushima
Abstract:
Graph Contrastive Learning (GCL) is unsupervised graph representation learning that can obtain useful representation of unknown nodes. The node representation can be utilized as features of downstream tasks. However, GCL is vulnerable to poisoning attacks as with existing learning models. A state-of-the-art defense cannot sufficiently negate adverse effects by poisoned graphs although such a defen…
▽ More
Graph Contrastive Learning (GCL) is unsupervised graph representation learning that can obtain useful representation of unknown nodes. The node representation can be utilized as features of downstream tasks. However, GCL is vulnerable to poisoning attacks as with existing learning models. A state-of-the-art defense cannot sufficiently negate adverse effects by poisoned graphs although such a defense introduces adversarial training in the GCL. To achieve further improvement, pruning adversarial edges is important. To the best of our knowledge, the feasibility remains unexplored in the GCL domain. In this paper, we propose a simple defense for GCL, EdgePruner. We focus on the fact that the state-of-the-art poisoning attack on GCL tends to mainly add adversarial edges to create poisoned graphs, which means that pruning edges is important to sanitize the graphs. Thus, EdgePruner prunes edges that contribute to minimizing the contrastive loss based on the node representation obtained after training on poisoned graphs by GCL. Furthermore, we focus on the fact that nodes with distinct features are connected by adversarial edges in poisoned graphs. Thus, we introduce feature similarity between neighboring nodes to help more appropriately determine adversarial edges. This similarity is helpful in further eliminating adverse effects from poisoned graphs on various datasets. Finally, EdgePruner outputs a graph that yields the minimum contrastive loss as the sanitized graph. Our results demonstrate that pruning adversarial edges is feasible on six datasets. EdgePruner can improve the accuracy of node classification under the attack by up to 5.55% compared with that of the state-of-the-art defense. Moreover, we show that EdgePruner is immune to an adaptive attack.
△ Less
Submitted 12 December, 2023;
originally announced December 2023.
-
VoteTRANS: Detecting Adversarial Text without Training by Voting on Hard Labels of Transformations
Authors:
Hoang-Quoc Nguyen-Son,
Seira Hidano,
Kazuhide Fukushima,
Shinsaku Kiyomoto,
Isao Echizen
Abstract:
Adversarial attacks reveal serious flaws in deep learning models. More dangerously, these attacks preserve the original meaning and escape human recognition. Existing methods for detecting these attacks need to be trained using original/adversarial data. In this paper, we propose detection without training by voting on hard labels from predictions of transformations, namely, VoteTRANS. Specificall…
▽ More
Adversarial attacks reveal serious flaws in deep learning models. More dangerously, these attacks preserve the original meaning and escape human recognition. Existing methods for detecting these attacks need to be trained using original/adversarial data. In this paper, we propose detection without training by voting on hard labels from predictions of transformations, namely, VoteTRANS. Specifically, VoteTRANS detects adversarial text by comparing the hard labels of input text and its transformation. The evaluation demonstrates that VoteTRANS effectively detects adversarial text across various state-of-the-art attacks, models, and datasets.
△ Less
Submitted 2 June, 2023;
originally announced June 2023.
-
Prior-mean-assisted Bayesian optimization application on FRIB Front-End tunning
Authors:
Kilean Hwang,
Tomofumi Maruta,
Alexander Plastun,
Kei Fukushima,
Tong Zhang,
Qiang Zhao,
Peter Ostroumov,
Yue Hao
Abstract:
Bayesian optimization~(BO) is often used for accelerator tuning due to its high sample efficiency. However, the computational scalability of training over large data-set can be problematic and the adoption of historical data in a computationally efficient way is not trivial. Here, we exploit a neural network model trained over historical data as a prior mean of BO for FRIB Front-End tuning.
Bayesian optimization~(BO) is often used for accelerator tuning due to its high sample efficiency. However, the computational scalability of training over large data-set can be problematic and the adoption of historical data in a computationally efficient way is not trivial. Here, we exploit a neural network model trained over historical data as a prior mean of BO for FRIB Front-End tuning.
△ Less
Submitted 11 November, 2022;
originally announced November 2022.
-
Node-wise Hardware Trojan Detection Based on Graph Learning
Authors:
Kento Hasegawa,
Kazuki Yamashita,
Seira Hidano,
Kazuhide Fukushima,
Kazuo Hashimoto,
Nozomu Togawa
Abstract:
In the fourth industrial revolution, securing the protection of the supply chain has become an ever-growing concern. One such cyber threat is a hardware Trojan (HT), a malicious modification to an IC. HTs are often identified in the hardware manufacturing process, but should be removed earlier, when the design is being specified. Machine learning-based HT detection in gate-level netlists is an eff…
▽ More
In the fourth industrial revolution, securing the protection of the supply chain has become an ever-growing concern. One such cyber threat is a hardware Trojan (HT), a malicious modification to an IC. HTs are often identified in the hardware manufacturing process, but should be removed earlier, when the design is being specified. Machine learning-based HT detection in gate-level netlists is an efficient approach to identify HTs at the early stage. However, feature-based modeling has limitations in discovering an appropriate set of HT features. We thus propose NHTD-GL in this paper, a novel node-wise HT detection method based on graph learning (GL). Given the formal analysis of HT features obtained from domain knowledge, NHTD-GL bridges the gap between graph representation learning and feature-based HT detection. The experimental results demonstrate that NHTD-GL achieves 0.998 detection accuracy and outperforms state-of-the-art node-wise HT detection methods. NHTD-GL extracts HT features without heuristic feature engineering.
△ Less
Submitted 15 March, 2022; v1 submitted 3 December, 2021;
originally announced December 2021.
-
SEPP: Similarity Estimation of Predicted Probabilities for Defending and Detecting Adversarial Text
Authors:
Hoang-Quoc Nguyen-Son,
Seira Hidano,
Kazuhide Fukushima,
Shinsaku Kiyomoto
Abstract:
There are two cases describing how a classifier processes input text, namely, misclassification and correct classification. In terms of misclassified texts, a classifier handles the texts with both incorrect predictions and adversarial texts, which are generated to fool the classifier, which is called a victim. Both types are misunderstood by the victim, but they can still be recognized by other c…
▽ More
There are two cases describing how a classifier processes input text, namely, misclassification and correct classification. In terms of misclassified texts, a classifier handles the texts with both incorrect predictions and adversarial texts, which are generated to fool the classifier, which is called a victim. Both types are misunderstood by the victim, but they can still be recognized by other classifiers. This induces large gaps in predicted probabilities between the victim and the other classifiers. In contrast, text correctly classified by the victim is often successfully predicted by the others and induces small gaps. In this paper, we propose an ensemble model based on similarity estimation of predicted probabilities (SEPP) to exploit the large gaps in the misclassified predictions in contrast to small gaps in the correct classification. SEPP then corrects the incorrect predictions of the misclassified texts. We demonstrate the resilience of SEPP in defending and detecting adversarial texts through different types of victim classifiers, classification tasks, and adversarial attacks.
△ Less
Submitted 12 October, 2021; v1 submitted 12 October, 2021;
originally announced October 2021.
-
Can a CNN trained on the Ising model detect the phase transition of the $q$-state Potts model?
Authors:
Kimihiko Fukushima,
Kazumitsu Sakai
Abstract:
Employing a deep convolutional neural network (deep CNN) trained on spin configurations of the 2D Ising model and the temperatures, we examine whether the deep CNN can detect the phase transition of the 2D $q$-state Potts model. To this end, we generate binarized images of spin configurations of the $q$-state Potts model ($q\ge 3$) by replacing the spin variables…
▽ More
Employing a deep convolutional neural network (deep CNN) trained on spin configurations of the 2D Ising model and the temperatures, we examine whether the deep CNN can detect the phase transition of the 2D $q$-state Potts model. To this end, we generate binarized images of spin configurations of the $q$-state Potts model ($q\ge 3$) by replacing the spin variables $\{0,1,\dots,\lfloor q/2\rfloor-1\}$ and $\{\lfloor q/2\rfloor,\dots,q-1\}$ with $\{0\}$ and $\{1\}$, respectively. Then, we input these images to the trained CNN to output the predicted temperatures. The binarized images of the $q$-state Potts model are entirely different from Ising spin configurations, particularly at the transition temperature. Moreover, our CNN model is not trained on the information about whether phases are ordered/disordered but is naively trained by Ising spin configurations labeled with temperatures at which they are generated. Nevertheless, the deep CNN can detect the transition point with high accuracy, regardless of the type of transition. We also find that, in the high-temperature region, the CNN outputs the temperature based on the internal energy, whereas, in the low-temperature region, the output depends on the magnetization and possibly the internal energy as well. However, in the vicinity of the transition point, the CNN may use more general factors to detect the transition point.
△ Less
Submitted 30 June, 2021; v1 submitted 8 April, 2021;
originally announced April 2021.
-
Extensive Studies of the Neutron Star Equation of State from the Deep Learning Inference with the Observational Data Augmentation
Authors:
Yuki Fujimoto,
Kenji Fukushima,
Koichi Murase
Abstract:
We discuss deep learning inference for the neutron star equation of state (EoS) using the real observational data of the mass and the radius. We make a quantitative comparison between the conventional polynomial regression and the neural network approach for the EoS parametrization. For our deep learning method to incorporate uncertainties in observation, we augment the training data with noise fl…
▽ More
We discuss deep learning inference for the neutron star equation of state (EoS) using the real observational data of the mass and the radius. We make a quantitative comparison between the conventional polynomial regression and the neural network approach for the EoS parametrization. For our deep learning method to incorporate uncertainties in observation, we augment the training data with noise fluctuations corresponding to observational uncertainties. Deduced EoSs can accommodate a weak first-order phase transition, and we make a histogram for likely first-order regions. We also find that our observational data augmentation has a byproduct to tame the overfitting behavior. To check the performance improved by the data augmentation, we set up a toy model as the simplest inference problem to recover a double-peaked function and monitor the validation loss. We conclude that the data augmentation could be a useful technique to evade the overfitting without tuning the neural network architecture such as inserting the dropout.
△ Less
Submitted 20 January, 2021;
originally announced January 2021.
-
Lattice-based Signcryption with Equality Test in Standard Model
Authors:
Huy Quoc Le,
Dung Hoang Duong,
Partha Sarathi Roy,
Willy Susilo,
Kazuhide Fukushima,
Shinsaku Kiyomoto
Abstract:
A signcryption, which is an integration of a public key encryption and a digital signature, can provide confidentiality and authenticity simultaneously. Additionally, a signcryption associated with equality test allows a third party (e.g., a cloud server) to check whether or not two ciphertexts are encrypted from the same message without knowing the message. This application plays an important rol…
▽ More
A signcryption, which is an integration of a public key encryption and a digital signature, can provide confidentiality and authenticity simultaneously. Additionally, a signcryption associated with equality test allows a third party (e.g., a cloud server) to check whether or not two ciphertexts are encrypted from the same message without knowing the message. This application plays an important role especially in computing on encrypted data. In this paper, we propose the first lattice-based signcryption scheme equipped with a solution to testing the message equality in the standard model. The proposed signcryption scheme is proven to be secure against insider attacks under the learning with errors assumption and the intractability of the short integer solution problem. As a by-product, we also show that some existing lattice-based signcryptions either is insecure or does not work correctly.
△ Less
Submitted 30 December, 2020;
originally announced December 2020.
-
Lattice-based public key encryption with equality test supporting flexible authorization in standard model
Authors:
Dung Hoang Duong,
Kazuhide Fukushima,
Shinsaku Kiyomoto,
Partha Sarathi Roy,
Arnaud Sipasseuth,
Willy Susilo
Abstract:
Public key encryption with equality test (PKEET) supports to check whether two ciphertexts encrypted under different public keys contain the same message or not. PKEET has many interesting applications such as keyword search on encrypted data, encrypted data partitioning for efficient encrypted data management, personal health record systems, spam filtering in encrypted email systems and so on. Ho…
▽ More
Public key encryption with equality test (PKEET) supports to check whether two ciphertexts encrypted under different public keys contain the same message or not. PKEET has many interesting applications such as keyword search on encrypted data, encrypted data partitioning for efficient encrypted data management, personal health record systems, spam filtering in encrypted email systems and so on. However, the PKEET scheme lacks an authorization mechanism for a user to control the comparison of its ciphertexts with others. In 2015, Ma et al. introduce the notion of PKEET with flexible authorization (PKEET-FA) which strengthens privacy protection. Since 2015, there are several follow-up works on PKEET-FA. But, all are secure in the random-oracle model. Moreover, all are vulnerable to quantum attacks. In this paper, we provide three constructions of quantum-safe PKEET-FA secure in the standard model. Proposed constructions are secure based on the hardness assumptions of integer lattices and ideal lattices. Finally, we implement the PKEET-FA scheme over ideal lattices.
△ Less
Submitted 9 May, 2020;
originally announced May 2020.
-
CCA2-secure Lattice-based Public Key Encryption with Equality Test in Standard Model
Authors:
Dung Hoang Duong,
Partha Sarathi Roy,
Willy Susilo,
Kazuhide Fukushima,
Shinsaku Kiyomoto,
Arnaud Sipasseuth
Abstract:
With the rapid growth of cloud storage and cloud computing services, many organisations and users choose to store the data on a cloud server for saving costs. However, due to security concerns, data of users would be encrypted before sending to the cloud. However, this hinders a problem of computation on encrypted data in the cloud, especially in the case of performing data matching in various med…
▽ More
With the rapid growth of cloud storage and cloud computing services, many organisations and users choose to store the data on a cloud server for saving costs. However, due to security concerns, data of users would be encrypted before sending to the cloud. However, this hinders a problem of computation on encrypted data in the cloud, especially in the case of performing data matching in various medical scenarios. Public key encryption with equality test (PKEET) is a powerful tool that allows the authorized cloud server to check whether two ciphertexts are generated by the same message. PKEET has then become a promising candidate for many practical applications like efficient data management on encrypted databases. Lee et al. (Information Sciences 2020) proposed a generic construction of PKEET schemes in the standard model and hence it is possible to yield the first instantiation of post-quantum PKEET schemes based on lattices. At ACISP 2019, Duong et al. proposed a direct construction of PKEET over integer lattices in the standard model. However, their scheme does not reach the CCA2-security. In this paper, we propose an efficient CCA2-secure PKEET scheme based on ideal lattices. In addition, we present a modification of the scheme by Duong et al. over integer lattices to attain the CCA2-security. Both schemes are proven secure in the standard model, and they enjoy the security in the upcoming quantum computer era.
△ Less
Submitted 31 January, 2021; v1 submitted 6 May, 2020;
originally announced May 2020.
-
Featuring the topology with the unsupervised machine learning
Authors:
Kenji Fukushima,
Shotaro Shiba Funai,
Hideaki Iida
Abstract:
Images of line drawings are generally composed of primitive elements. One of the most fundamental elements to characterize images is the topology; line segments belong to a category different from closed circles, and closed circles with different winding degrees are nonequivalent. We investigate images with nontrivial winding using the unsupervised machine learning. We build an autoencoder model w…
▽ More
Images of line drawings are generally composed of primitive elements. One of the most fundamental elements to characterize images is the topology; line segments belong to a category different from closed circles, and closed circles with different winding degrees are nonequivalent. We investigate images with nontrivial winding using the unsupervised machine learning. We build an autoencoder model with a combination of convolutional and fully connected neural networks. We confirm that compressed data filtered from the trained model retain more than 90% of correct information on the topology, evidencing that image clustering from the unsupervised learning features the topology.
△ Less
Submitted 1 August, 2019;
originally announced August 2019.
