Showing 1–16 of 16 results for author: Kurri, G R

Maximal Guesswork Leakage

Authors: Gowtham R. Kurri, Malhar Managoli, Vinod M. Prabhakaran

Abstract: We introduce the study of information leakage through \emph{guesswork}, the minimum expected number of guesses required to guess a random variable. In particular, we define \emph{maximal guesswork leakage} as the multiplicative decrease, upon observing $Y$, of the guesswork of a randomized function of $X$, maximized over all such randomized functions. We also study a pointwise form of the leakage… ▽ More We introduce the study of information leakage through \emph{guesswork}, the minimum expected number of guesses required to guess a random variable. In particular, we define \emph{maximal guesswork leakage} as the multiplicative decrease, upon observing $Y$, of the guesswork of a randomized function of $X$, maximized over all such randomized functions. We also study a pointwise form of the leakage which captures the leakage due to the release of a single realization of $Y$. We also study these two notions of leakage with oblivious (or memoryless) guessing. We obtain closed-form expressions for all these leakage measures, with the exception of one. Specifically, we are able to obtain closed-form expression for maximal guesswork leakage for the binary erasure source only; deriving expressions for arbitrary sources appears challenging. Some of the consequences of our results are -- a connection between guesswork and differential privacy and a new operational interpretation to maximal $α$-leakage in terms of guesswork. △ Less

Submitted 4 May, 2024; originally announced May 2024.

Comments: 6 pages. Extended version of a paper accepted to ISIT 2024

Addressing GAN Training Instabilities via Tunable Classification Losses

Authors: Monica Welfert, Gowtham R. Kurri, Kyle Otstot, Lalitha Sankar

Abstract: Generative adversarial networks (GANs), modeled as a zero-sum game between a generator (G) and a discriminator (D), allow generating synthetic data with formal guarantees. Noting that D is a classifier, we begin by reformulating the GAN value function using class probability estimation (CPE) losses. We prove a two-way correspondence between CPE loss GANs and $f$-GANs which minimize $f$-divergences… ▽ More Generative adversarial networks (GANs), modeled as a zero-sum game between a generator (G) and a discriminator (D), allow generating synthetic data with formal guarantees. Noting that D is a classifier, we begin by reformulating the GAN value function using class probability estimation (CPE) losses. We prove a two-way correspondence between CPE loss GANs and $f$-GANs which minimize $f$-divergences. We also show that all symmetric $f$-divergences are equivalent in convergence. In the finite sample and model capacity setting, we define and obtain bounds on estimation and generalization errors. We specialize these results to $α$-GANs, defined using $α$-loss, a tunable CPE loss family parametrized by $α\in(0,\infty]$. We next introduce a class of dual-objective GANs to address training instabilities of GANs by modeling each player's objective using $α$-loss to obtain $(α_D,α_G)$-GANs. We show that the resulting non-zero sum game simplifies to minimizing an $f$-divergence under appropriate conditions on $(α_D,α_G)$. Generalizing this dual-objective formulation using CPE losses, we define and obtain upper bounds on an appropriately defined estimation error. Finally, we highlight the value of tuning $(α_D,α_G)$ in alleviating training instabilities for the synthetic 2D Gaussian mixture ring as well as the large publicly available Celeb-A and LSUN Classroom image datasets. △ Less

Submitted 27 October, 2023; originally announced October 2023.

Comments: arXiv admin note: text overlap with arXiv:2302.14320

Unifying Privacy Measures via Maximal $(α,β)$-Leakage (M$α$beL)

Authors: Atefeh Gilani, Gowtham R. Kurri, Oliver Kosut, Lalitha Sankar

Abstract: We introduce a family of information leakage measures called maximal $(α,β)$-leakage (M$α$beL), parameterized by real numbers $α$ and $β$ greater than or equal to 1. The measure is formalized via an operational definition involving an adversary guessing an unknown (randomized) function of the data given the released data. We obtain a simplified computable expression for the measure and show that i… ▽ More We introduce a family of information leakage measures called maximal $(α,β)$-leakage (M$α$beL), parameterized by real numbers $α$ and $β$ greater than or equal to 1. The measure is formalized via an operational definition involving an adversary guessing an unknown (randomized) function of the data given the released data. We obtain a simplified computable expression for the measure and show that it satisfies several basic properties such as monotonicity in $β$ for a fixed $α$, non-negativity, data processing inequalities, and additivity over independent releases. We highlight the relevance of this family by showing that it bridges several known leakage measures, including maximal $α$-leakage $(β=1)$, maximal leakage $(α=\infty,β=1)$, local differential privacy (LDP) $(α=\infty,β=\infty)$, and local Renyi differential privacy (LRDP) $(α=β)$, thereby giving an operational interpretation to local Renyi differential privacy. We also study a conditional version of M$α$beL on leveraging which we recover differential privacy and Renyi differential privacy. A new variant of LRDP, which we call maximal Renyi leakage, appears as a special case of M$α$beL for $α=\infty$ that smoothly tunes between maximal leakage ($β=1$) and LDP ($β=\infty$). Finally, we show that a vector form of the maximal Renyi leakage relaxes differential privacy under Gaussian and Laplacian mechanisms. △ Less

Submitted 4 April, 2024; v1 submitted 14 April, 2023; originally announced April 2023.

$(α_D,α_G)$-GANs: Addressing GAN Training Instabilities via Dual Objectives

Authors: Monica Welfert, Kyle Otstot, Gowtham R. Kurri, Lalitha Sankar

Abstract: In an effort to address the training instabilities of GANs, we introduce a class of dual-objective GANs with different value functions (objectives) for the generator (G) and discriminator (D). In particular, we model each objective using $α$-loss, a tunable classification loss, to obtain $(α_D,α_G)$-GANs, parameterized by $(α_D,α_G)\in (0,\infty]^2$. For sufficiently large number of samples and ca… ▽ More In an effort to address the training instabilities of GANs, we introduce a class of dual-objective GANs with different value functions (objectives) for the generator (G) and discriminator (D). In particular, we model each objective using $α$-loss, a tunable classification loss, to obtain $(α_D,α_G)$-GANs, parameterized by $(α_D,α_G)\in (0,\infty]^2$. For sufficiently large number of samples and capacities for G and D, we show that the resulting non-zero sum game simplifies to minimizing an $f$-divergence under appropriate conditions on $(α_D,α_G)$. In the finite sample and capacity setting, we define estimation error to quantify the gap in the generator's performance relative to the optimal setting with infinite samples and obtain upper bounds on this error, showing it to be order optimal under certain conditions. Finally, we highlight the value of tuning $(α_D,α_G)$ in alleviating training instabilities for the synthetic 2D Gaussian mixture ring and the Stacked MNIST datasets. △ Less

Submitted 3 May, 2023; v1 submitted 28 February, 2023; originally announced February 2023.

Comments: Extended version of a paper accepted to ISIT 2023

An Alphabet of Leakage Measures

Authors: Atefeh Gilani, Gowtham R. Kurri, Oliver Kosut, Lalitha Sankar

Abstract: We introduce a family of information leakage measures called maximal $α,β$-leakage, parameterized by real numbers $α$ and $β$. The measure is formalized via an operational definition involving an adversary guessing an unknown function of the data given the released data. We obtain a simple, computable expression for the measure and show that it satisfies several basic properties such as monotonici… ▽ More We introduce a family of information leakage measures called maximal $α,β$-leakage, parameterized by real numbers $α$ and $β$. The measure is formalized via an operational definition involving an adversary guessing an unknown function of the data given the released data. We obtain a simple, computable expression for the measure and show that it satisfies several basic properties such as monotonicity in $β$ for a fixed $α$, non-negativity, data processing inequalities, and additivity over independent releases. Finally, we highlight the relevance of this family by showing that it bridges several known leakage measures, including maximal $α$-leakage $(β=1)$, maximal leakage $(α=\infty,β=1)$, local differential privacy $(α=\infty,β=\infty)$, and local Renyi differential privacy $(α=β)$. △ Less

Submitted 28 November, 2022; originally announced November 2022.

An Operational Approach to Information Leakage via Generalized Gain Functions

Authors: Gowtham R. Kurri, Lalitha Sankar, Oliver Kosut

Abstract: We introduce a \emph{gain function} viewpoint of information leakage by proposing \emph{maximal $g$-leakage}, a rich class of operationally meaningful leakage measures that subsumes recently introduced leakage measures -- {maximal leakage} and {maximal $α$-leakage}. In maximal $g$-leakage, the gain of an adversary in guessing an unknown random variable is measured using a {gain function} applied t… ▽ More We introduce a \emph{gain function} viewpoint of information leakage by proposing \emph{maximal $g$-leakage}, a rich class of operationally meaningful leakage measures that subsumes recently introduced leakage measures -- {maximal leakage} and {maximal $α$-leakage}. In maximal $g$-leakage, the gain of an adversary in guessing an unknown random variable is measured using a {gain function} applied to the probability of correctly guessing. In particular, maximal $g$-leakage captures the multiplicative increase, upon observing $Y$, in the expected gain of an adversary in guessing a randomized function of $X$, maximized over all such randomized functions. We also consider the scenario where an adversary can make multiple attempts to guess the randomized function of interest. We show that maximal leakage is an upper bound on maximal $g$-leakage under multiple guesses, for any non-negative gain function $g$. We obtain a closed-form expression for maximal $g$-leakage under multiple guesses for a class of concave gain functions. We also study maximal $g$-leakage measure for a specific class of gain functions related to the $α$-loss. In particular, we first completely characterize the minimal expected $α$-loss under multiple guesses and analyze how the corresponding leakage measure is affected with the number of guesses. Finally, we study two variants of maximal $g$-leakage depending on the type of adversary and obtain closed-form expressions for them, which do not depend on the particular gain function considered as long as it satisfies some mild regularity conditions. We do this by develo** a variational characterization for the Rényi divergence of order infinity which naturally generalizes the definition of pointwise maximal leakage to incorporate arbitrary gain functions. △ Less

Submitted 7 December, 2023; v1 submitted 28 September, 2022; originally announced September 2022.

Comments: 27 pages, 1 Figure. New results are added. Some results of this paper were presented at ISIT 2021 and ISIT 2022

$α$-GAN: Convergence and Estimation Guarantees

Authors: Gowtham R. Kurri, Monica Welfert, Tyler Sypherd, Lalitha Sankar

Abstract: We prove a two-way correspondence between the min-max optimization of general CPE loss function GANs and the minimization of associated $f$-divergences. We then focus on $α$-GAN, defined via the $α$-loss, which interpolates several GANs (Hellinger, vanilla, Total Variation) and corresponds to the minimization of the Arimoto divergence. We show that the Arimoto divergences induced by $α$-GAN equiva… ▽ More We prove a two-way correspondence between the min-max optimization of general CPE loss function GANs and the minimization of associated $f$-divergences. We then focus on $α$-GAN, defined via the $α$-loss, which interpolates several GANs (Hellinger, vanilla, Total Variation) and corresponds to the minimization of the Arimoto divergence. We show that the Arimoto divergences induced by $α$-GAN equivalently converge, for all $α\in \mathbb{R}_{>0}\cup\{\infty\}$. However, under restricted learning models and finite samples, we provide estimation bounds which indicate diverse GAN behavior as a function of $α$. Finally, we present empirical results on a toy dataset that highlight the practical utility of tuning the $α$ hyperparameter. △ Less

Submitted 12 May, 2022; originally announced May 2022.

Comments: Extended version of a paper accepted to ISIT 2022. 12 pages, 7 figures

A Variational Formula for Infinity-Rényi Divergence with Applications to Information Leakage

Authors: Gowtham R. Kurri, Oliver Kosut, Lalitha Sankar

Abstract: We present a variational characterization for the Rényi divergence of order infinity. Our characterization is related to guessing: the objective functional is a ratio of maximal expected values of a gain function applied to the probability of correctly guessing an unknown random variable. An important aspect of our variational characterization is that it remains agnostic to the particular gain fun… ▽ More We present a variational characterization for the Rényi divergence of order infinity. Our characterization is related to guessing: the objective functional is a ratio of maximal expected values of a gain function applied to the probability of correctly guessing an unknown random variable. An important aspect of our variational characterization is that it remains agnostic to the particular gain function considered, as long as it satisfies some regularity conditions. Also, we define two variants of a tunable measure of information leakage, the maximal $α$-leakage, and obtain closed-form expressions for these information measures by leveraging our variational characterization. △ Less

Submitted 2 May, 2022; v1 submitted 12 February, 2022; originally announced February 2022.

Comments: Accepted to ISIT 2022, 6 pages

Evaluating Multiple Guesses by an Adversary via a Tunable Loss Function

Authors: Gowtham R. Kurri, Oliver Kosut, Lalitha Sankar

Abstract: We consider a problem of guessing, wherein an adversary is interested in knowing the value of the realization of a discrete random variable $X$ on observing another correlated random variable $Y$. The adversary can make multiple (say, $k$) guesses. The adversary's guessing strategy is assumed to minimize $α$-loss, a class of tunable loss functions parameterized by $α$. It has been shown before tha… ▽ More We consider a problem of guessing, wherein an adversary is interested in knowing the value of the realization of a discrete random variable $X$ on observing another correlated random variable $Y$. The adversary can make multiple (say, $k$) guesses. The adversary's guessing strategy is assumed to minimize $α$-loss, a class of tunable loss functions parameterized by $α$. It has been shown before that this loss function captures well known loss functions including the exponential loss ($α=1/2$), the log-loss ($α=1$) and the $0$-$1$ loss ($α=\infty$). We completely characterize the optimal adversarial strategy and the resulting expected $α$-loss, thereby recovering known results for $α=\infty$. We define an information leakage measure from the $k$-guesses setup and derive a condition under which the leakage is unchanged from a single guess. △ Less

Submitted 19 August, 2021; originally announced August 2021.

Comments: 6 pages

Realizing GANs via a Tunable Loss Function

Authors: Gowtham R. Kurri, Tyler Sypherd, Lalitha Sankar

Abstract: We introduce a tunable GAN, called $α$-GAN, parameterized by $α\in (0,\infty]$, which interpolates between various $f$-GANs and Integral Probability Metric based GANs (under constrained discriminator set). We construct $α$-GAN using a supervised loss function, namely, $α$-loss, which is a tunable loss function capturing several canonical losses. We show that $α$-GAN is intimately related to the Ar… ▽ More We introduce a tunable GAN, called $α$-GAN, parameterized by $α\in (0,\infty]$, which interpolates between various $f$-GANs and Integral Probability Metric based GANs (under constrained discriminator set). We construct $α$-GAN using a supervised loss function, namely, $α$-loss, which is a tunable loss function capturing several canonical losses. We show that $α$-GAN is intimately related to the Arimoto divergence, which was first proposed by Österriecher (1996), and later studied by Liese and Vajda (2006). We also study the convergence properties of $α$-GAN. We posit that the holistic understanding that $α$-GAN introduces will have practical benefits of addressing both the issues of vanishing gradients and mode collapse. △ Less

Submitted 18 October, 2021; v1 submitted 9 June, 2021; originally announced June 2021.

Comments: Extended version of a paper accepted to ITW 2021. 8 pages, 2 figures

Multiple Access Channel Simulation

Authors: Gowtham R. Kurri, Viswanathan Ramachandran, Sibi Raj B. Pillai, Vinod M. Prabhakaran

Abstract: We study the problem of simulating a two-user multiple-access channel (MAC) over a multiple access network of noiseless links. Two encoders observe independent and identically distributed (i.i.d.) copies of a source random variable each, while a decoder observes i.i.d. copies of a side-information random variable. There are rate-limited noiseless communication links between each encoder and the de… ▽ More We study the problem of simulating a two-user multiple-access channel (MAC) over a multiple access network of noiseless links. Two encoders observe independent and identically distributed (i.i.d.) copies of a source random variable each, while a decoder observes i.i.d. copies of a side-information random variable. There are rate-limited noiseless communication links between each encoder and the decoder, and there is independent pairwise shared randomness between all the three possible pairs of nodes. The decoder has to output approximately i.i.d. copies of another random variable jointly distributed with the two sources and the side information. We are interested in the rate tuples which permit this simulation. This setting can be thought of as a multi-terminal generalization of the point-to-point channel simulation problem studied by Bennett et al. (2002) and Cuff (2013). When the pairwise shared randomness between the encoders is absent, the setting reduces to a special case of MAC simulation using another MAC studied by Haddadpour et al.~(2013). We establish that the presence of encoder shared randomness can strictly improve the communication rate requirements. We first show that the inner bound derived from Haddadpour et al.~(2013) is tight when the sources at the encoders are conditionally independent given the side-information at the decoder. This result recovers the existing results on point-to-point channel simulation and function computation over such multi-terminal networks. We then explicitly compute the communication rate regions for an example both with and without the encoder shared randomness and demonstrate that its presence strictly reduces the communication rates. Inner and outer bounds for the general case are also obtained. △ Less

Submitted 16 June, 2022; v1 submitted 23 February, 2021; originally announced February 2021.

Comments: 30 pages, 3 figures

Secure Computation to Hide Functions of Inputs

Authors: Gowtham R. Kurri, Vinod M. Prabhakaran

Abstract: We consider a two-user secure computation problem in which Alice and Bob communicate interactively in order to compute some deterministic functions of the inputs. The privacy requirement is that each user should not learn any additional information about a function of the inputs other than what can be inferred from its own input and output. For the distribution-free setting, i.e., when the protoco… ▽ More We consider a two-user secure computation problem in which Alice and Bob communicate interactively in order to compute some deterministic functions of the inputs. The privacy requirement is that each user should not learn any additional information about a function of the inputs other than what can be inferred from its own input and output. For the distribution-free setting, i.e., when the protocol must be correct and private for any joint input distribution, we completely characterize the set of all securely computable functions. When privacy is required only against Bob who computes a function based on a single transmission from Alice, we show that asymptotically secure computability is equivalent to perfectly secure computability. Separately, we consider an eavesdropper who has access to all the communication and should not learn any information about some function of the inputs (possibly different from the functions to be computed by the users) and show that interaction may be necessary for secure computation. △ Less

Submitted 11 May, 2020; originally announced May 2020.

Comments: Extended version of a paper to appear at International Symposium on Information Theory (ISIT) 2020. 7 pages, 3 figures

Coordination Through Shared Randomness

Authors: Gowtham R. Kurri, Vinod M. Prabhakaran, Anand D. Sarwate

Abstract: We study a distributed sampling problem where a set of processors want to output (approximately) independent and identically distributed samples from a joint distribution with the help of a common message from a coordinator. Each processor has access to a subset of sources from a set of independent sources of "shared" randomness. We consider two cases -- in the "omniscient coordinator setting", th… ▽ More We study a distributed sampling problem where a set of processors want to output (approximately) independent and identically distributed samples from a joint distribution with the help of a common message from a coordinator. Each processor has access to a subset of sources from a set of independent sources of "shared" randomness. We consider two cases -- in the "omniscient coordinator setting", the coordinator has access to all these sources of shared randomness, while in the "oblivious coordinator setting", it has access to none. All processors and the coordinator may privately randomize. In the omniscient coordinator setting, when the subsets at the processors are disjoint (individually shared randomness model), we characterize the rate of communication required from the coordinator to the processors over a multicast link. For the two-processor case, the optimal rate matches a special case of relaxed Wyner's common information proposed by Gastpar and Sula (2019), thereby providing an operational meaning to the latter. We also give an upper bound on the communication rate for the "randomness-on-the-forehead" model where each processor observes all but one source of randomness and we give an achievable strategy for the general case where the processors have access to arbitrary subsets of sources of randomness. Also, we consider a more general model where the processors observe components of correlated sources (with the coordinator observing all the components), where we characterize the communication rate when all the processors wish to output the same random sequence. In the oblivious coordinator setting, we completely characterize the trade-off region between the communication and shared randomness rates for the general case where the processors have access to arbitrary subsets of sources of randomness. △ Less

Submitted 17 June, 2021; v1 submitted 22 August, 2019; originally announced August 2019.

Comments: 27 pages, 7 figures. Some results of this paper were presented at ISIT 2018 and ITW 2019. This paper subsumes arXiv:1805.03193

Optimal Communication Rates and Combinatorial Properties for Common Randomness Generation

Authors: Yanjun Han, Kedar Tatwawadi, Gowtham R. Kurri, Zhengqing Zhou, Vinod M. Prabhakaran, Tsachy Weissman

Abstract: We study common randomness generation problems where $n$ players aim to generate same sequences of random coin flips where some subsets of the players share an independent common coin which can be tossed multiple times, and there is a publicly seen blackboard through which the players communicate with each other. We provide a tight representation of the optimal communication rates via linear progr… ▽ More We study common randomness generation problems where $n$ players aim to generate same sequences of random coin flips where some subsets of the players share an independent common coin which can be tossed multiple times, and there is a publicly seen blackboard through which the players communicate with each other. We provide a tight representation of the optimal communication rates via linear programming, and more importantly, propose explicit algorithms for the optimal distributed simulation for a wide class of hypergraphs. In particular, the optimal communication rate in complete hypergraphs is still achievable in sparser hypergraphs containing a path-connected cycle-free cluster of topologically connected components. Some key steps in analyzing the upper bounds rely on two different definitions of connectivity in hypergraphs, which may be of independent interest. △ Less

Submitted 6 October, 2021; v1 submitted 5 April, 2019; originally announced April 2019.

Comments: 17 pages, 10 figures

Interactive Secure Function Computation

Authors: Deepesh Data, Gowtham R. Kurri, Jithin Ravi, Vinod M. Prabhakaran

Abstract: We consider interactive computation of randomized functions between two users with the following privacy requirement: the interaction should not reveal to either user any extra information about the other user's input and output other than what can be inferred from the user's own input and output. We also consider the case where privacy is required against only one of the users. For both cases, we… ▽ More We consider interactive computation of randomized functions between two users with the following privacy requirement: the interaction should not reveal to either user any extra information about the other user's input and output other than what can be inferred from the user's own input and output. We also consider the case where privacy is required against only one of the users. For both cases, we give single-letter expressions for feasibility and optimal rates of communication. Then we discuss the role of common randomness and interaction in both privacy settings. We also study perfectly secure non-interactive computation when only one of the users computes a randomized function based on a single transmission from the other user. We characterize randomized functions which can be perfectly securely computed in this model and obtain tight bounds on the optimal message lengths in all the privacy settings. △ Less

Submitted 9 March, 2020; v1 submitted 10 December, 2018; originally announced December 2018.

Comments: 30 pages. Revised based on comments from the reviewers

Coordination Using Individually Shared Randomness

Authors: Gowtham R. Kurri, Vinod M. Prabhakaran, Anand D. Sarwate

Abstract: Two processors output correlated sequences using the help of a coordinator with whom they individually share independent randomness. For the case of unlimited shared randomness, we characterize the rate of communication required from the coordinator to the processors over a broadcast link. We also give an achievable trade-off between the communication and shared randomness rates. Two processors output correlated sequences using the help of a coordinator with whom they individually share independent randomness. For the case of unlimited shared randomness, we characterize the rate of communication required from the coordinator to the processors over a broadcast link. We also give an achievable trade-off between the communication and shared randomness rates. △ Less

Submitted 8 May, 2018; originally announced May 2018.

Comments: Extended version of a paper accepted for presentation at ISIT 2018. 8 pages, 3 figures