-
Towards Credential-based Device Registration in DApps for DePINs with ZKPs
Authors:
Jonathan Heiss,
Fernando Castillo,
Xinxin Fan
Abstract:
Decentralized Physical Infrastructure Networks (DePINS) are secured and governed by blockchains but beyond crypto-economic incentives, they lack measures to establish trust in participating devices and their services. The verification of relevant device credentials during device registration helps to overcome this problem. However, on-chain verification in decentralized applications (dApp) disclos…
▽ More
Decentralized Physical Infrastructure Networks (DePINS) are secured and governed by blockchains but beyond crypto-economic incentives, they lack measures to establish trust in participating devices and their services. The verification of relevant device credentials during device registration helps to overcome this problem. However, on-chain verification in decentralized applications (dApp) discloses potentially confidential device attributes whereas off-chain verification introduces undesirable trust assumptions. In this paper, we propose a credential-based device registration (CDR) mechanism that verifies device credentials on the blockchain and leverages zero-knowledge proofs (ZKP) to protect confidential device attributes from being disclosed. We characterize CDR for DePINs, present a general system model, and technically evaluate CDR using zkSNARKs with Groth16 and Marlin. Our experiments give first insights into performance impacts and reveal a tradeoff between the applied proof systems.
△ Less
Submitted 27 June, 2024;
originally announced June 2024.
-
Servicifying zk-SNARKs Execution for Verifiable Off-chain Computations
Authors:
Alvaro Alonso Domenech,
Jonathan Heiss,
Stefan Tai
Abstract:
Zk-SNARKs help scale blockchains with Verifiable Off-chain Computations (VOC). zk-SNARK DSL toolkits are key when designing arithmetic circuits but fall short of automating the subsequent proof-generation step in an automated manner. We emphasize the need for portability, interoperability, and manageability in VOC-based solutions and introduce a Proving Service that is designed to provide a scalab…
▽ More
Zk-SNARKs help scale blockchains with Verifiable Off-chain Computations (VOC). zk-SNARK DSL toolkits are key when designing arithmetic circuits but fall short of automating the subsequent proof-generation step in an automated manner. We emphasize the need for portability, interoperability, and manageability in VOC-based solutions and introduce a Proving Service that is designed to provide a scalable and reusable solution for generating zk-SNARK proofs leveraging clouds.
△ Less
Submitted 25 April, 2024;
originally announced April 2024.
-
End-to-End Verifiable Decentralized Federated Learning
Authors:
Chaehyeon Lee,
Jonathan Heiss,
Stefan Tai,
James Won-Ki Hong
Abstract:
Verifiable decentralized federated learning (FL) systems combining blockchains and zero-knowledge proofs (ZKP) make the computational integrity of local learning and global aggregation verifiable across workers. However, they are not end-to-end: data can still be corrupted prior to the learning. In this paper, we propose a verifiable decentralized FL system for end-to-end integrity and authenticit…
▽ More
Verifiable decentralized federated learning (FL) systems combining blockchains and zero-knowledge proofs (ZKP) make the computational integrity of local learning and global aggregation verifiable across workers. However, they are not end-to-end: data can still be corrupted prior to the learning. In this paper, we propose a verifiable decentralized FL system for end-to-end integrity and authenticity of data and computation extending verifiability to the data source. Addressing an inherent conflict of confidentiality and transparency, we introduce a two-step proving and verification (2PV) method that we apply to central system procedures: a registration workflow that enables non-disclosing verification of device certificates and a learning workflow that extends existing blockchain and ZKP-based FL systems through non-disclosing data authenticity proofs. Our evaluation on a prototypical implementation demonstrates the technical feasibility with only marginal overheads to state-of-the-art solutions.
△ Less
Submitted 19 April, 2024;
originally announced April 2024.
-
Machine Learning-Powered Combinatorial Clock Auction
Authors:
Ermis Soumalias,
Jakob Weissteiner,
Jakob Heiss,
Sven Seuken
Abstract:
We study the design of iterative combinatorial auctions (ICAs). The main challenge in this domain is that the bundle space grows exponentially in the number of items. To address this, several papers have recently proposed machine learning (ML)-based preference elicitation algorithms that aim to elicit only the most important information from bidders. However, from a practical point of view, the ma…
▽ More
We study the design of iterative combinatorial auctions (ICAs). The main challenge in this domain is that the bundle space grows exponentially in the number of items. To address this, several papers have recently proposed machine learning (ML)-based preference elicitation algorithms that aim to elicit only the most important information from bidders. However, from a practical point of view, the main shortcoming of this prior work is that those designs elicit bidders' preferences via value queries (i.e., ``What is your value for the bundle $\{A,B\}$?''). In most real-world ICA domains, value queries are considered impractical, since they impose an unrealistically high cognitive burden on bidders, which is why they are not used in practice. In this paper, we address this shortcoming by designing an ML-powered combinatorial clock auction that elicits information from the bidders only via demand queries (i.e., ``At prices $p$, what is your most preferred bundle of items?''). We make two key technical contributions: First, we present a novel method for training an ML model on demand queries. Second, based on those trained ML models, we introduce an efficient method for determining the demand query with the highest clearing potential, for which we also provide a theoretical foundation. We experimentally evaluate our ML-based demand query mechanism in several spectrum auction domains and compare it against the most established real-world ICA: the combinatorial clock auction (CCA). Our mechanism significantly outperforms the CCA in terms of efficiency in all domains, it achieves higher efficiency in a significantly reduced number of rounds, and, using linear prices, it exhibits vastly higher clearing potential. Thus, with this paper we bridge the gap between research and practice and propose the first practical ML-powered ICA.
△ Less
Submitted 28 March, 2024; v1 submitted 20 August, 2023;
originally announced August 2023.
-
Extending Path-Dependent NJ-ODEs to Noisy Observations and a Dependent Observation Framework
Authors:
William Andersson,
Jakob Heiss,
Florian Krach,
Josef Teichmann
Abstract:
The Path-Dependent Neural Jump Ordinary Differential Equation (PD-NJ-ODE) is a model for predicting continuous-time stochastic processes with irregular and incomplete observations. In particular, the method learns optimal forecasts given irregularly sampled time series of incomplete past observations. So far the process itself and the coordinate-wise observation times were assumed to be independen…
▽ More
The Path-Dependent Neural Jump Ordinary Differential Equation (PD-NJ-ODE) is a model for predicting continuous-time stochastic processes with irregular and incomplete observations. In particular, the method learns optimal forecasts given irregularly sampled time series of incomplete past observations. So far the process itself and the coordinate-wise observation times were assumed to be independent and observations were assumed to be noiseless. In this work we discuss two extensions to lift these restrictions and provide theoretical guarantees as well as empirical examples for them. In particular, we can lift the assumption of independence by extending the theory to much more realistic settings of conditional independence without any need to change the algorithm. Moreover, we introduce a new loss function, which allows us to deal with noisy observations and explain why the previously used loss function did not lead to a consistent estimator.
△ Less
Submitted 5 February, 2024; v1 submitted 24 July, 2023;
originally announced July 2023.
-
How (Implicit) Regularization of ReLU Neural Networks Characterizes the Learned Function -- Part II: the Multi-D Case of Two Layers with Random First Layer
Authors:
Jakob Heiss,
Josef Teichmann,
Hanna Wutte
Abstract:
Randomized neural networks (randomized NNs), where only the terminal layer's weights are optimized constitute a powerful model class to reduce computational time in training the neural network model. At the same time, these models generalize surprisingly well in various regression and classification tasks. In this paper, we give an exact macroscopic characterization (i.e., a characterization in fu…
▽ More
Randomized neural networks (randomized NNs), where only the terminal layer's weights are optimized constitute a powerful model class to reduce computational time in training the neural network model. At the same time, these models generalize surprisingly well in various regression and classification tasks. In this paper, we give an exact macroscopic characterization (i.e., a characterization in function space) of the generalization behavior of randomized, shallow NNs with ReLU activation (RSNs). We show that RSNs correspond to a generalized additive model (GAM)-typed regression in which infinitely many directions are considered: the infinite generalized additive model (IGAM). The IGAM is formalized as solution to an optimization problem in function space for a specific regularization functional and a fairly general loss. This work is an extension to multivariate NNs of prior work, where we showed how wide RSNs with ReLU activation behave like spline regression under certain conditions and if the input is one-dimensional.
△ Less
Submitted 20 March, 2023;
originally announced March 2023.
-
Non-Disclosing Credential On-chaining for Blockchain-based Decentralized Applications
Authors:
Jonathan Heiss,
Robert Muth,
Frank Pallas,
Stefan Tai
Abstract:
Many service systems rely on verifiable identity-related information of their users. Manipulation and unwanted exposure of this privacy-relevant information, however, must at the same time be prevented and avoided. Peer-to-peer blockchain-based decentralization with a smart contract-based execution model and verifiable off-chain computations leveraging zero-knowledge proofs promise to provide the…
▽ More
Many service systems rely on verifiable identity-related information of their users. Manipulation and unwanted exposure of this privacy-relevant information, however, must at the same time be prevented and avoided. Peer-to-peer blockchain-based decentralization with a smart contract-based execution model and verifiable off-chain computations leveraging zero-knowledge proofs promise to provide the basis for next-generation, non-disclosing credential management solutions. In this paper, we propose a novel credential on-chaining system that ensures blockchain-based transparency while preserving pseudonymity. We present a general model compliant to the W3C verifiable credential recommendation and demonstrate how it can be applied to solve existing problems that require computational identity-related attribute verification. Our zkSNARKs-based reference implementation and evaluation show that, compared to related approaches based on, e.g., CL-signatures, our approach provides significant performance advantages and more flexible proof mechanisms, underpinning our vision of increasingly decentralized, transparent, and trustworthy service systems.
△ Less
Submitted 20 September, 2022;
originally announced September 2022.
-
Bayesian Optimization-based Combinatorial Assignment
Authors:
Jakob Weissteiner,
Jakob Heiss,
Julien Siems,
Sven Seuken
Abstract:
We study the combinatorial assignment domain, which includes combinatorial auctions and course allocation. The main challenge in this domain is that the bundle space grows exponentially in the number of items. To address this, several papers have recently proposed machine learning-based preference elicitation algorithms that aim to elicit only the most important information from agents. However, t…
▽ More
We study the combinatorial assignment domain, which includes combinatorial auctions and course allocation. The main challenge in this domain is that the bundle space grows exponentially in the number of items. To address this, several papers have recently proposed machine learning-based preference elicitation algorithms that aim to elicit only the most important information from agents. However, the main shortcoming of this prior work is that it does not model a mechanism's uncertainty over values for not yet elicited bundles. In this paper, we address this shortcoming by presenting a Bayesian optimization-based combinatorial assignment (BOCA) mechanism. Our key technical contribution is to integrate a method for capturing model uncertainty into an iterative combinatorial auction mechanism. Concretely, we design a new method for estimating an upper uncertainty bound that can be used to define an acquisition function to determine the next query to the agents. This enables the mechanism to properly explore (and not just exploit) the bundle space during its preference elicitation phase. We run computational experiments in several spectrum auction domains to evaluate BOCA's performance. Our results show that BOCA achieves higher allocative efficiency than state-of-the-art approaches.
△ Less
Submitted 13 March, 2023; v1 submitted 31 August, 2022;
originally announced August 2022.
-
Advancing Blockchain-based Federated Learning through Verifiable Off-chain Computations
Authors:
Jonathan Heiss,
Elias Grünewald,
Nikolas Haimerl,
Stefan Schulte,
Stefan Tai
Abstract:
Federated learning may be subject to both global aggregation attacks and distributed poisoning attacks. Blockchain technology along with incentive and penalty mechanisms have been suggested to counter these. In this paper, we explore verifiable off-chain computations using zero-knowledge proofs as an alternative to incentive and penalty mechanisms in blockchain-based federated learning. In our sol…
▽ More
Federated learning may be subject to both global aggregation attacks and distributed poisoning attacks. Blockchain technology along with incentive and penalty mechanisms have been suggested to counter these. In this paper, we explore verifiable off-chain computations using zero-knowledge proofs as an alternative to incentive and penalty mechanisms in blockchain-based federated learning. In our solution, learning nodes, in addition to their computational duties, act as off-chain provers submitting proofs to attest computational correctness of parameters that can be verified on the blockchain. We demonstrate and evaluate our solution through a health monitoring use case and proof-of-concept implementation leveraging the ZoKrates language and tools for smart contract-based on-chain model management. Our research introduces verifiability of correctness of learning processes, thus advancing blockchain-based federated learning.
△ Less
Submitted 23 June, 2022;
originally announced June 2022.
-
A Network Approach to Atomic Spectra
Authors:
David Wellnitz,
Armin Kekić,
Julian Heiss,
Michael Gertz,
Matthias Weidemüller,
Andreas Spitz
Abstract:
Network science provides a universal framework for modeling complex systems, contrasting the reductionist approach generally adopted in physics. In a prototypical study, we utilize network models created from spectroscopic data of atoms to predict microscopic properties of the underlying physical system. For simple atoms such as helium, an a posteriori inspection of spectroscopic network communiti…
▽ More
Network science provides a universal framework for modeling complex systems, contrasting the reductionist approach generally adopted in physics. In a prototypical study, we utilize network models created from spectroscopic data of atoms to predict microscopic properties of the underlying physical system. For simple atoms such as helium, an a posteriori inspection of spectroscopic network communities reveals the emergence of quantum numbers and symmetries. For more complex atoms such as thorium, finer network hierarchies suggest additional microscopic symmetries or configurations. Link prediction yields a quantitative ranking of yet unknown atomic transitions, offering opportunities to discover new spectral lines in a well-controlled manner. Our work promotes a genuine bi-directional exchange of methodology between network science and physics, and presents new perspectives for the study of atomic spectra.
△ Less
Submitted 9 February, 2022;
originally announced February 2022.
-
How Infinitely Wide Neural Networks Can Benefit from Multi-task Learning -- an Exact Macroscopic Characterization
Authors:
Jakob Heiss,
Josef Teichmann,
Hanna Wutte
Abstract:
In practice, multi-task learning (through learning features shared among tasks) is an essential property of deep neural networks (NNs). While infinite-width limits of NNs can provide good intuition for their generalization behavior, the well-known infinite-width limits of NNs in the literature (e.g., neural tangent kernels) assume specific settings in which wide ReLU-NNs behave like shallow Gaussi…
▽ More
In practice, multi-task learning (through learning features shared among tasks) is an essential property of deep neural networks (NNs). While infinite-width limits of NNs can provide good intuition for their generalization behavior, the well-known infinite-width limits of NNs in the literature (e.g., neural tangent kernels) assume specific settings in which wide ReLU-NNs behave like shallow Gaussian Processes with a fixed kernel. Consequently, in such settings, these NNs lose their ability to benefit from multi-task learning in the infinite-width limit. In contrast, we prove that optimizing wide ReLU neural networks with at least one hidden layer using L2-regularization on the parameters promotes multi-task learning due to representation-learning - also in the limiting regime where the network width tends to infinity. We present an exact quantitative characterization of this infinite width limit in an appropriate function space that neatly describes multi-task learning.
△ Less
Submitted 20 October, 2022; v1 submitted 31 December, 2021;
originally announced December 2021.
-
Trustworthy Pre-Processing of Sensor Data in Data On-chaining Workflows for Blockchain-based IoT Applications
Authors:
Jonathan Heiss,
Anselm Busse,
Stefan Tai
Abstract:
Prior to provisioning sensor data to smart contracts, a pre-processing of the data on intermediate off-chain nodes is often necessary. When doing so, originally constructed cryptographic signatures cannot be verified on-chain anymore. This exposes an opportunity for undetected manipulation and presents a problem for applications in the Internet of Things where trustworthy sensor data is required o…
▽ More
Prior to provisioning sensor data to smart contracts, a pre-processing of the data on intermediate off-chain nodes is often necessary. When doing so, originally constructed cryptographic signatures cannot be verified on-chain anymore. This exposes an opportunity for undetected manipulation and presents a problem for applications in the Internet of Things where trustworthy sensor data is required on-chain. In this paper, we propose trustworthy pre-processing as enabler for end-to-end sensor data integrity in data on-chaining workflows. We define requirements for trustworthy pre-processing, present a model and common workflow for data on-chaining, select off-chain computation utilizing Zero-knowledge Proofs (ZKPs) and Trusted Execution Environments (TEEs) as promising solution approaches, and discuss both our proof-of-concept implementations and initial experimental, comparative evaluation results. The importance of trustworthy pre-processing and principle solution approaches are presented, addressing the major problem of end-to-end sensor data integrity in blockchain-based IoT applications.
△ Less
Submitted 29 October, 2021;
originally announced October 2021.
-
Monotone-Value Neural Networks: Exploiting Preference Monotonicity in Combinatorial Assignment
Authors:
Jakob Weissteiner,
Jakob Heiss,
Julien Siems,
Sven Seuken
Abstract:
Many important resource allocation problems involve the combinatorial assignment of items, e.g., auctions or course allocation. Because the bundle space grows exponentially in the number of items, preference elicitation is a key challenge in these domains. Recently, researchers have proposed ML-based mechanisms that outperform traditional mechanisms while reducing preference elicitation costs for…
▽ More
Many important resource allocation problems involve the combinatorial assignment of items, e.g., auctions or course allocation. Because the bundle space grows exponentially in the number of items, preference elicitation is a key challenge in these domains. Recently, researchers have proposed ML-based mechanisms that outperform traditional mechanisms while reducing preference elicitation costs for agents. However, one major shortcoming of the ML algorithms that were used is their disregard of important prior knowledge about agents' preferences. To address this, we introduce monotone-value neural networks (MVNNs), which are designed to capture combinatorial valuations, while enforcing monotonicity and normality. On a technical level, we prove that our MVNNs are universal in the class of monotone and normalized value functions, and we provide a mixed-integer linear program (MILP) formulation to make solving MVNN-based winner determination problems (WDPs) practically feasible. We evaluate our MVNNs experimentally in spectrum auction domains. Our results show that MVNNs improve the prediction performance, they yield state-of-the-art allocative efficiency in the auction, and they also reduce the run-time of the WDPs. Our code is available on GitHub: https://github.com/marketdesignresearch/MVNN.
△ Less
Submitted 11 March, 2023; v1 submitted 30 September, 2021;
originally announced September 2021.
-
NOMU: Neural Optimization-based Model Uncertainty
Authors:
Jakob Heiss,
Jakob Weissteiner,
Hanna Wutte,
Sven Seuken,
Josef Teichmann
Abstract:
We study methods for estimating model uncertainty for neural networks (NNs) in regression. To isolate the effect of model uncertainty, we focus on a noiseless setting with scarce training data. We introduce five important desiderata regarding model uncertainty that any method should satisfy. However, we find that established benchmarks often fail to reliably capture some of these desiderata, even…
▽ More
We study methods for estimating model uncertainty for neural networks (NNs) in regression. To isolate the effect of model uncertainty, we focus on a noiseless setting with scarce training data. We introduce five important desiderata regarding model uncertainty that any method should satisfy. However, we find that established benchmarks often fail to reliably capture some of these desiderata, even those that are required by Bayesian theory. To address this, we introduce a new approach for capturing model uncertainty for NNs, which we call Neural Optimization-based Model Uncertainty (NOMU). The main idea of NOMU is to design a network architecture consisting of two connected sub-NNs, one for model prediction and one for model uncertainty, and to train it using a carefully-designed loss function. Importantly, our design enforces that NOMU satisfies our five desiderata. Due to its modular architecture, NOMU can provide model uncertainty for any given (previously trained) NN if given access to its training data. We evaluate NOMU in various regressions tasks and noiseless Bayesian optimization (BO) with costly evaluations. In regression, NOMU performs at least as well as state-of-the-art methods. In BO, NOMU even outperforms all considered benchmarks.
△ Less
Submitted 11 March, 2023; v1 submitted 26 February, 2021;
originally announced February 2021.
-
How Implicit Regularization of ReLU Neural Networks Characterizes the Learned Function -- Part I: the 1-D Case of Two Layers with Random First Layer
Authors:
Jakob Heiss,
Josef Teichmann,
Hanna Wutte
Abstract:
In this paper, we consider one dimensional (shallow) ReLU neural networks in which weights are chosen randomly and only the terminal layer is trained. First, we mathematically show that for such networks L2-regularized regression corresponds in function space to regularizing the estimate's second derivative for fairly general loss functionals. For least squares regression, we show that the trained…
▽ More
In this paper, we consider one dimensional (shallow) ReLU neural networks in which weights are chosen randomly and only the terminal layer is trained. First, we mathematically show that for such networks L2-regularized regression corresponds in function space to regularizing the estimate's second derivative for fairly general loss functionals. For least squares regression, we show that the trained network converges to the smooth spline interpolation of the training data as the number of hidden nodes tends to infinity. Moreover, we derive a novel correspondence between the early stopped gradient descent (without any explicit regularization of the weights) and the smoothing spline regression.
△ Less
Submitted 4 October, 2023; v1 submitted 7 November, 2019;
originally announced November 2019.
-
Mechanical model for gravity
Authors:
Jonny Heiss
Abstract:
Assuming the existence of a Multidirectional Homogeneous and Constant Shower of Elementary Particles (MHCSEP) traveling at light speed in space, several basic laws of physics are derived mainly by geometrical considerations. When placing two bodies in space, obstruction of the MHCSEP creates an attractive force among them that coincides, for two bodies, with Newton's law of gravity, generating a…
▽ More
Assuming the existence of a Multidirectional Homogeneous and Constant Shower of Elementary Particles (MHCSEP) traveling at light speed in space, several basic laws of physics are derived mainly by geometrical considerations. When placing two bodies in space, obstruction of the MHCSEP creates an attractive force among them that coincides, for two bodies, with Newton's law of gravity, generating a mechanical explanation for gravity.
△ Less
Submitted 21 May, 2000;
originally announced May 2000.
