Showing 1–16 of 16 results for author: Wetzel, S

Closed-Form Interpretation of Neural Network Classifiers with Symbolic Regression Gradients

Authors: Sebastian Johann Wetzel

Abstract: I introduce a unified framework for interpreting neural network classifiers tailored toward automated scientific discovery. In contrast to neural network-based regression, for classification, it is in general impossible to find a one-to-one map** from the neural network to a symbolic equation even if the neural network itself bases its classification on a quantity that can be written as a closed… ▽ More I introduce a unified framework for interpreting neural network classifiers tailored toward automated scientific discovery. In contrast to neural network-based regression, for classification, it is in general impossible to find a one-to-one map** from the neural network to a symbolic equation even if the neural network itself bases its classification on a quantity that can be written as a closed-form equation. In this paper, I embed a trained neural network into an equivalence class of classifying functions that base their decisions on the same quantity. I interpret neural networks by finding an intersection between this equivalence class and human-readable equations defined by the search space of symbolic regression. The approach is not limited to classifiers or full neural networks and can be applied to arbitrary neurons in hidden layers or latent spaces or to simplify the process of interpreting neural network regressors. △ Less

Submitted 10 January, 2024; originally announced January 2024.

Twin Neural Network Improved k-Nearest Neighbor Regression

Authors: Sebastian J. Wetzel

Abstract: Twin neural network regression is trained to predict differences between regression targets rather than the targets themselves. A solution to the original regression problem can be obtained by ensembling predicted differences between the targets of an unknown data point and multiple known anchor data points. Choosing the anchors to be the nearest neighbors of the unknown data point leads to a neur… ▽ More Twin neural network regression is trained to predict differences between regression targets rather than the targets themselves. A solution to the original regression problem can be obtained by ensembling predicted differences between the targets of an unknown data point and multiple known anchor data points. Choosing the anchors to be the nearest neighbors of the unknown data point leads to a neural network-based improvement of k-nearest neighbor regression. This algorithm is shown to outperform both neural networks and k-nearest neighbor regression on small to medium-sized data sets. △ Less

Submitted 1 October, 2023; originally announced October 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2301.01383

Efficient Privacy-Preserving Approximation of the Kidney Exchange Problem

Authors: Malte Breuer, Ulrike Meyer, Susanne Wetzel

Abstract: The kidney exchange problem (KEP) seeks to find possible exchanges among pairs of patients and their incompatible kidney donors while meeting specific optimization criteria such as maximizing the overall number of possible transplants. Recently, several privacy-preserving protocols for solving the KEP have been proposed. However, the protocols known to date lack scalability in practice since the K… ▽ More The kidney exchange problem (KEP) seeks to find possible exchanges among pairs of patients and their incompatible kidney donors while meeting specific optimization criteria such as maximizing the overall number of possible transplants. Recently, several privacy-preserving protocols for solving the KEP have been proposed. However, the protocols known to date lack scalability in practice since the KEP is an NP-complete problem. We address this issue by proposing a novel privacy-preserving protocol which computes an approximate solution for the KEP that scales well for the large numbers of patient-donor pairs encountered in practice. As opposed to prior work on privacy-preserving kidney exchange, our protocol is generic w.r.t.\ the security model that can be employed. Compared to the most efficient privacy-preserving protocols for kidney exchange existing to date, our protocol is entirely data oblivious and it exhibits a far superior run time performance. As a second contribution, we use a real-world data set to simulate the application of our protocol as part of a kidney exchange platform, where patient-donor pairs register and de-register over time, and thereby determine its approximation quality in a real-world setting. △ Less

Submitted 1 February, 2024; v1 submitted 27 February, 2023; originally announced February 2023.

Comments: Accepted at ACM Asia Conference on Computer and Communications Security (ASIA CCS 2024)

How to get the most out of Twinned Regression Methods

Authors: Sebastian J. Wetzel

Abstract: Twinned regression methods are designed to solve the dual problem to the original regression problem, predicting differences between regression targets rather then the targets themselves. A solution to the original regression problem can be obtained by ensembling predicted differences between the targets of an unknown data point and multiple known anchor data points. We explore different aspects o… ▽ More Twinned regression methods are designed to solve the dual problem to the original regression problem, predicting differences between regression targets rather then the targets themselves. A solution to the original regression problem can be obtained by ensembling predicted differences between the targets of an unknown data point and multiple known anchor data points. We explore different aspects of twinned regression methods: (1) We decompose different steps in twinned regression algorithms and examine their contributions to the final performance, (2) We examine the intrinsic ensemble quality, (3) We combine twin neural network regression with k-nearest neighbor regression to design a more accurate and efficient regression method, and (4) we develop a simplified semi-supervised regression scheme. △ Less

Submitted 3 January, 2023; originally announced January 2023.

Solving the Kidney Exchange Problem Using Privacy-Preserving Integer Programming (Updated and Extended Version)

Authors: Malte Breuer, Pascal Hein, Leonardo Pompe, Ben Temme, Ulrike Meyer, Susanne Wetzel

Abstract: The kidney exchange problem (KEP) is to find a constellation of exchanges that maximizes the number of transplants that can be carried out for a set of pairs of patients with kidney disease and their incompatible donors. Recently, this problem has been tackled from a privacy perspective in order to protect the sensitive medical data of patients and donors and to decrease the potential for manipula… ▽ More The kidney exchange problem (KEP) is to find a constellation of exchanges that maximizes the number of transplants that can be carried out for a set of pairs of patients with kidney disease and their incompatible donors. Recently, this problem has been tackled from a privacy perspective in order to protect the sensitive medical data of patients and donors and to decrease the potential for manipulation of the computing of the exchanges. However, the proposed approaches to date either only compute an approximative solution to the KEP or they suffer from a huge decrease in performance. In this paper, we suggest a novel privacy-preserving protocol that computes an exact solution to the KEP and significantly outperforms the other existing exact approaches. Our novel protocol is based on Integer Programming which is the most efficient method for solving the KEP in the non privacy-preserving case. We achieve an improved performance compared to the privacy-preserving approaches known to date by extending the output of the ideal functionality to include the termination decisions of the underlying algorithm. We implement our protocol in the SMPC benchmarking framework MP-SPDZ and compare its performance to the existing protocols for solving the KEP. In this extended version of our paper, we also evaluate whether and if so how much information can be inferred from the extended output of the ideal functionality. △ Less

Submitted 23 October, 2023; v1 submitted 24 August, 2022; originally announced August 2022.

Comments: This is the updated and extended version of the work published in 19th Annual International Conference on Privacy, Security and Trust (PST2022), August 22-24, 2022, Fredericton, Canada / Virtual Conference, https://doi.org/10.1109/PST55820.2022.9851968

Unsupervised Learning of Rydberg Atom Array Phase Diagram with Siamese Neural Networks

Authors: Zakaria Patel, Ejaaz Merali, Sebastian J. Wetzel

Abstract: We introduce an unsupervised machine learning method based on Siamese Neural Networks (SNN) to detect phase boundaries. This method is applied to Monte-Carlo simulations of Ising-type systems and Rydberg atom arrays. In both cases the SNN reveals phase boundaries consistent with prior research. The combination of leveraging the power of feed-forward neural networks, unsupervised learning and the a… ▽ More We introduce an unsupervised machine learning method based on Siamese Neural Networks (SNN) to detect phase boundaries. This method is applied to Monte-Carlo simulations of Ising-type systems and Rydberg atom arrays. In both cases the SNN reveals phase boundaries consistent with prior research. The combination of leveraging the power of feed-forward neural networks, unsupervised learning and the ability to learn about multiple phases without knowing about their existence provides a powerful method to explore new and unknown phases of matter. △ Less

Submitted 19 May, 2022; v1 submitted 9 May, 2022; originally announced May 2022.

Privacy-Preserving Maximum Matching on General Graphs and its Application to Enable Privacy-Preserving Kidney Exchange

Authors: Malte Breuer, Ulrike Meyer, Susanne Wetzel

Abstract: To this day, there are still some countries where the exchange of kidneys between multiple incompatible patient-donor pairs is restricted by law. Typically, legal regulations in this context are put in place to prohibit coercion and manipulation in order to prevent a market for organ trade. Yet, in countries where kidney exchange is practiced, existing platforms to facilitate such exchanges genera… ▽ More To this day, there are still some countries where the exchange of kidneys between multiple incompatible patient-donor pairs is restricted by law. Typically, legal regulations in this context are put in place to prohibit coercion and manipulation in order to prevent a market for organ trade. Yet, in countries where kidney exchange is practiced, existing platforms to facilitate such exchanges generally lack sufficient privacy mechanisms. In this paper, we propose a privacy-preserving protocol for kidney exchange that not only addresses the privacy problem of existing platforms but also is geared to lead the way in overcoming legal issues in those countries where kidney exchange is still not practiced. In our approach, we use the concept of secret sharing to distribute the medical data of patients and donors among a set of computing peers in a privacy-preserving fashion. These computing peers then execute our new Secure Multi-Party Computation (SMPC) protocol among each other to determine an optimal set of kidney exchanges. As part of our new protocol, we devise a privacy-preserving solution to the maximum matching problem on general graphs. We have implemented the protocol in the SMPC benchmarking framework MP-SPDZ and provide a comprehensive performance evaluation. Furthermore, we analyze the practicality of our protocol when used in a dynamic setting (where patients and donors arrive and depart over time) based on a data set from the United Network for Organ Sharing. △ Less

Submitted 2 June, 2022; v1 submitted 17 January, 2022; originally announced January 2022.

Comments: This is the extended version of the paper that appeared in the 12th ACM Conference on Data and Application Security and Privacy (CODASPY'22), April 24-26, 2022, Baltimore-Washington DC Area, United States, https://doi.org/10.1145/3508398.3511509

Journal ref: 12th ACM Conference on Data and Application Security and Privacy 2022 (CODASPY'22)

Introducing a Framework to Enable Anonymous Secure Multi-Party Computation in Practice (Extended Version)

Authors: Malte Breuer, Ulrike Meyer, Susanne Wetzel

Abstract: Secure Multi-Party Computation (SMPC) allows a set of parties to securely compute a functionality in a distributed fashion without the need for any trusted external party. Usually, it is assumed that the parties know each other and have already established authenticated channels among each other. However, in practice the parties sometimes must stay anonymous. In this paper, we conceptualize a fram… ▽ More Secure Multi-Party Computation (SMPC) allows a set of parties to securely compute a functionality in a distributed fashion without the need for any trusted external party. Usually, it is assumed that the parties know each other and have already established authenticated channels among each other. However, in practice the parties sometimes must stay anonymous. In this paper, we conceptualize a framework that enables the repeated execution of an SMPC protocol for a given functionality such that the parties can keep their participation in the protocol executions private and at the same time be sure that only authorized parties may take part in a protocol execution. We identify the security properties that an implementation of our framework must meet and introduce a first implementation of the framework that achieves these properties. △ Less

Submitted 19 January, 2023; v1 submitted 3 November, 2021; originally announced November 2021.

Comments: This is the extended version of the paper that appeared in the 18th International Conference on Privacy, Security and Trust (PST 2021) (DOI: 10.1109/PST52912.2021.9647793)

Twin Neural Network Regression is a Semi-Supervised Regression Algorithm

Authors: Sebastian J. Wetzel, Roger G. Melko, Isaac Tamblyn

Abstract: Twin neural network regression (TNNR) is a semi-supervised regression algorithm, it can be trained on unlabelled data points as long as other, labelled anchor data points, are present. TNNR is trained to predict differences between the target values of two different data points rather than the targets themselves. By ensembling predicted differences between the targets of an unseen data point and a… ▽ More Twin neural network regression (TNNR) is a semi-supervised regression algorithm, it can be trained on unlabelled data points as long as other, labelled anchor data points, are present. TNNR is trained to predict differences between the target values of two different data points rather than the targets themselves. By ensembling predicted differences between the targets of an unseen data point and all training data points, it is possible to obtain a very accurate prediction for the original regression problem. Since any loop of predicted differences should sum to zero, loops can be supplied to the training data, even if the data points themselves within loops are unlabelled. Semi-supervised training improves TNNR performance, which is already state of the art, significantly. △ Less

Submitted 10 June, 2021; originally announced June 2021.

Twin Neural Network Regression

Authors: Sebastian J. Wetzel, Kevin Ryczko, Roger G. Melko, Isaac Tamblyn

Abstract: We introduce twin neural network (TNN) regression. This method predicts differences between the target values of two different data points rather than the targets themselves. The solution of a traditional regression problem is then obtained by averaging over an ensemble of all predicted differences between the targets of an unseen data point and all training data points. Whereas ensembles are norm… ▽ More We introduce twin neural network (TNN) regression. This method predicts differences between the target values of two different data points rather than the targets themselves. The solution of a traditional regression problem is then obtained by averaging over an ensemble of all predicted differences between the targets of an unseen data point and all training data points. Whereas ensembles are normally costly to produce, TNN regression intrinsically creates an ensemble of predictions of twice the size of the training set while only training a single neural network. Since ensembles have been shown to be more accurate than single models this property naturally transfers to TNN regression. We show that TNNs are able to compete or yield more accurate predictions for different data sets, compared to other state-of-the-art methods. Furthermore, TNN regression is constrained by self-consistency conditions. We find that the violation of these conditions provides an estimate for the prediction uncertainty. △ Less

Submitted 29 December, 2020; originally announced December 2020.

Logic Guided Genetic Algorithms

Authors: Dhananjay Ashok, Joseph Scott, Sebastian Wetzel, Maysum Panju, Vijay Ganesh

Abstract: We present a novel Auxiliary Truth enhanced Genetic Algorithm (GA) that uses logical or mathematical constraints as a means of data augmentation as well as to compute loss (in conjunction with the traditional MSE), with the aim of increasing both data efficiency and accuracy of symbolic regression (SR) algorithms. Our method, logic-guided genetic algorithm (LGGA), takes as input a set of labelled… ▽ More We present a novel Auxiliary Truth enhanced Genetic Algorithm (GA) that uses logical or mathematical constraints as a means of data augmentation as well as to compute loss (in conjunction with the traditional MSE), with the aim of increasing both data efficiency and accuracy of symbolic regression (SR) algorithms. Our method, logic-guided genetic algorithm (LGGA), takes as input a set of labelled data points and auxiliary truths (ATs) (mathematical facts known a priori about the unknown function the regressor aims to learn) and outputs a specially generated and curated dataset that can be used with any SR method. Three key insights underpin our method: first, SR users often know simple ATs about the function they are trying to learn. Second, whenever an SR system produces a candidate equation inconsistent with these ATs, we can compute a counterexample to prove the inconsistency, and further, this counterexample may be used to augment the dataset and fed back to the SR system in a corrective feedback loop. Third, the value addition of these ATs is that their use in both the loss function and the data augmentation process leads to better rates of convergence, accuracy, and data efficiency. We evaluate LGGA against state-of-the-art SR tools, namely, Eureqa and TuringBot on 16 physics equations from "The Feynman Lectures on Physics" book. We find that using these SR tools in conjunction with LGGA results in them solving up to 30.0% more equations, needing only a fraction of the amount of data compared to the same tool without LGGA, i.e., resulting in up to a 61.9% improvement in data efficiency. △ Less

Submitted 21 October, 2020; originally announced October 2020.

A Privacy-Preserving Protocol for the Kidney Exchange Problem

Authors: Malte Breuer, Ulrike Meyer, Susanne Wetzel, Anja Mühlfeld

Abstract: Kidney donations from living donors form an attractive alternative to long waiting times on a list for a post-mortem donation. However, even if a living donor for a given patient is found, the donor's kidney might not meet the patient's medical requirements. If several patients are in this position, they may be able to exchange donors in a cyclic fashion. Current algorithmic approaches for determi… ▽ More Kidney donations from living donors form an attractive alternative to long waiting times on a list for a post-mortem donation. However, even if a living donor for a given patient is found, the donor's kidney might not meet the patient's medical requirements. If several patients are in this position, they may be able to exchange donors in a cyclic fashion. Current algorithmic approaches for determining such exchange cycles neglect the privacy requirements of donors and patients as they require their medical data to be centrally collected and evaluated. In this paper, we present the first distributed privacy-preserving protocol for kidney exchange that ensures the correct computing of the exchange cycles while at the same time protecting the privacy of the patients' sensitive medical data. We prove correctness and security of the new protocol and evaluate its practical performance. △ Less

Submitted 23 September, 2020; originally announced September 2020.

Journal ref: Proceedings of the 19th Workshop on Privacy in the Electronic Society (WPES 2020)

Discovering Symmetry Invariants and Conserved Quantities by Interpreting Siamese Neural Networks

Authors: Sebastian J. Wetzel, Roger G. Melko, Joseph Scott, Maysum Panju, Vijay Ganesh

Abstract: In this paper, we introduce interpretable Siamese Neural Networks (SNN) for similarity detection to the field of theoretical physics. More precisely, we apply SNNs to events in special relativity, the transformation of electromagnetic fields, and the motion of particles in a central potential. In these examples, the SNNs learn to identify datapoints belonging to the same events, field configuratio… ▽ More In this paper, we introduce interpretable Siamese Neural Networks (SNN) for similarity detection to the field of theoretical physics. More precisely, we apply SNNs to events in special relativity, the transformation of electromagnetic fields, and the motion of particles in a central potential. In these examples, the SNNs learn to identify datapoints belonging to the same events, field configurations, or trajectory of motion. It turns out that in the process of learning which datapoints belong to the same event or field configuration, these SNNs also learn the relevant symmetry invariants and conserved quantities. These SNNs are highly interpretable, which enables us to reveal the symmetry invariants and conserved quantities without prior knowledge. △ Less

Submitted 25 August, 2020; v1 submitted 9 March, 2020; originally announced March 2020.

Journal ref: Phys. Rev. Research 2, 033499 (2020)

Spectral Reconstruction with Deep Neural Networks

Authors: Lukas Kades, Jan M. Pawlowski, Alexander Rothkopf, Manuel Scherzer, Julian M. Urban, Sebastian J. Wetzel, Nicolas Wink, Felix P. G. Ziegler

Abstract: We explore artificial neural networks as a tool for the reconstruction of spectral functions from imaginary time Green's functions, a classic ill-conditioned inverse problem. Our ansatz is based on a supervised learning framework in which prior knowledge is encoded in the training data and the inverse transformation manifold is explicitly parametrised through a neural network. We systematically in… ▽ More We explore artificial neural networks as a tool for the reconstruction of spectral functions from imaginary time Green's functions, a classic ill-conditioned inverse problem. Our ansatz is based on a supervised learning framework in which prior knowledge is encoded in the training data and the inverse transformation manifold is explicitly parametrised through a neural network. We systematically investigate this novel reconstruction approach, providing a detailed analysis of its performance on physically motivated mock data, and compare it to established methods of Bayesian inference. The reconstruction accuracy is found to be at least comparable, and potentially superior in particular at larger noise levels. We argue that the use of labelled training data in a supervised setting and the freedom in defining an optimisation objective are inherent advantages of the present approach and may lead to significant improvements over state-of-the-art methods in the future. Potential directions for further research are discussed in detail. △ Less

Submitted 1 February, 2021; v1 submitted 10 May, 2019; originally announced May 2019.

Comments: 20 pages, 16 figures

Journal ref: Phys. Rev. D 102, 096001 (2020)

The INSuRE Project: CAE-Rs Collaborate to Engage Students in Cybersecurity Research

Authors: Alan Sherman, M. Dark, A. Chan, R. Chong, T. Morris, L. Oliva, J. Springer, B. Thuraisingham, C. Vatcher, R. Verma, S. Wetzel

Abstract: Since fall 2012, several National Centers of Academic Excellence in Cyber Defense Research (CAE-Rs) fielded a collaborative course to engage students in solving applied cybersecurity research problems. We describe our experiences with this Information Security Research and Education (INSuRE) research collaborative. We explain how we conducted our project-based research course, give examples of stu… ▽ More Since fall 2012, several National Centers of Academic Excellence in Cyber Defense Research (CAE-Rs) fielded a collaborative course to engage students in solving applied cybersecurity research problems. We describe our experiences with this Information Security Research and Education (INSuRE) research collaborative. We explain how we conducted our project-based research course, give examples of student projects, and discuss the outcomes and lessons learned. △ Less

Submitted 26 March, 2017; originally announced March 2017.

Comments: A shorter version of this paper has been submitted to IEEE Security and Privacy

Unsupervised learning of phase transitions: from principal component analysis to variational autoencoders

Authors: Sebastian Johann Wetzel

Abstract: We employ unsupervised machine learning techniques to learn latent parameters which best describe states of the two-dimensional Ising model and the three-dimensional XY model. These methods range from principal component analysis to artificial neural network based variational autoencoders. The states are sampled using a Monte-Carlo simulation above and below the critical temperature. We find that… ▽ More We employ unsupervised machine learning techniques to learn latent parameters which best describe states of the two-dimensional Ising model and the three-dimensional XY model. These methods range from principal component analysis to artificial neural network based variational autoencoders. The states are sampled using a Monte-Carlo simulation above and below the critical temperature. We find that the predicted latent parameters correspond to the known order parameters. The latent representation of the states of the models in question are clustered, which makes it possible to identify phases without prior knowledge of their existence or the underlying Hamiltonian. Furthermore, we find that the reconstruction loss function can be used as a universal identifier for phase transitions. △ Less

Submitted 12 March, 2017; v1 submitted 7 March, 2017; originally announced March 2017.

Comments: corrected typos

Journal ref: Phys. Rev. E 96, 022140 (2017)