Showing 1–47 of 47 results for author: Oechtering, T J

Information Density Bounds for Privacy

Authors: Sara Saeidian, Leonhard Grosse, Parastoo Sadeghi, Mikael Skoglund, Tobias J. Oechtering

Abstract: This paper explores the implications of guaranteeing privacy by imposing a lower bound on the information density between the private and the public data. We introduce an operationally meaningful privacy measure called pointwise maximal cost (PMC) and demonstrate that imposing an upper bound on PMC is equivalent to enforcing a lower bound on the information density. PMC quantifies the information… ▽ More This paper explores the implications of guaranteeing privacy by imposing a lower bound on the information density between the private and the public data. We introduce an operationally meaningful privacy measure called pointwise maximal cost (PMC) and demonstrate that imposing an upper bound on PMC is equivalent to enforcing a lower bound on the information density. PMC quantifies the information leakage about a secret to adversaries who aim to minimize non-negative cost functions after observing the outcome of a privacy mechanism. When restricted to finite alphabets, PMC can equivalently be defined as the information leakage to adversaries aiming to minimize the probability of incorrectly guessing randomized functions of the secret. We study the properties of PMC and apply it to standard privacy mechanisms to demonstrate its practical relevance. Through a detailed examination, we connect PMC with other privacy measures that impose upper or lower bounds on the information density. Our results highlight that lower bounding the information density is a more stringent requirement than upper bounding it. Overall, our work significantly bridges the gaps in understanding the relationships between various privacy frameworks and provides insights for selecting a suitable framework for a given application. △ Less

Submitted 1 July, 2024; originally announced July 2024.

Multi-Task Private Semantic Communication

Authors: Amirreza Zamani, Sajad Daei, Tobias J. Oechtering, Mikael Skoglund

Abstract: We study a multi-task private semantic communication problem, in which an encoder has access to an information source arbitrarily correlated with some latent private data. A user has $L$ tasks with priorities. The encoder designs a message to be revealed which is called the semantic of the information source. Due to the privacy constraints the semantic can not be disclosed directly and the encoder… ▽ More We study a multi-task private semantic communication problem, in which an encoder has access to an information source arbitrarily correlated with some latent private data. A user has $L$ tasks with priorities. The encoder designs a message to be revealed which is called the semantic of the information source. Due to the privacy constraints the semantic can not be disclosed directly and the encoder adds noise to produce disclosed data. The goal is to design the disclosed data that maximizes the weighted sum of the utilities achieved by the user while satisfying a privacy constraint on the private data. In this work, we first consider a single-task scenario and design the added noise utilizing various methods including the extended versions of the Functional Representation Lemma, Strong Functional Representation Lemma, and separation technique. We then study the multi-task scenario and derive a simple design of the source semantics. We show that in the multi-task scenario the main problem can be divided into multiple parallel single-task problems. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Chained Information-Theoretic bounds and Tight Regret Rate for Linear Bandit Problems

Authors: Amaury Gouverneur, Borja Rodríguez-Gálvez, Tobias J. Oechtering, Mikael Skoglund

Abstract: This paper studies the Bayesian regret of a variant of the Thompson-Sampling algorithm for bandit problems. It builds upon the information-theoretic framework of [Russo and Van Roy, 2015] and, more specifically, on the rate-distortion analysis from [Dong and Van Roy, 2020], where they proved a bound with regret rate of $O(d\sqrt{T \log(T)})$ for the $d$-dimensional linear bandit setting. We focus… ▽ More This paper studies the Bayesian regret of a variant of the Thompson-Sampling algorithm for bandit problems. It builds upon the information-theoretic framework of [Russo and Van Roy, 2015] and, more specifically, on the rate-distortion analysis from [Dong and Van Roy, 2020], where they proved a bound with regret rate of $O(d\sqrt{T \log(T)})$ for the $d$-dimensional linear bandit setting. We focus on bandit problems with a metric action space and, using a chaining argument, we establish new bounds that depend on the metric entropy of the action space for a variant of Thompson-Sampling. Under suitable continuity assumption of the rewards, our bound offers a tight rate of $O(d\sqrt{T})$ for $d$-dimensional linear bandit problems. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Comments: 15 pages: 8 of main text and 7 of appendices

Quantifying Privacy via Information Density

Authors: Leonhard Grosse, Sara Saeidian, Parastoo Sadeghi, Tobias J. Oechtering, Mikael Skoglund

Abstract: We examine the relationship between privacy metrics that utilize information density to measure information leakage between a private and a disclosed random variable. Firstly, we prove that bounding the information density from above or below in turn implies a lower or upper bound on the information density, respectively. Using this result, we establish new relationships between local information… ▽ More We examine the relationship between privacy metrics that utilize information density to measure information leakage between a private and a disclosed random variable. Firstly, we prove that bounding the information density from above or below in turn implies a lower or upper bound on the information density, respectively. Using this result, we establish new relationships between local information privacy, asymmetric local information privacy, pointwise maximal leakage and local differential privacy. We further provide applications of these relations to privacy mechanism design. Furthermore, we provide statements showing the equivalence between a lower bound on information density and risk-averse adversaries. More specifically, we prove an equivalence between a guessing framework and a cost-function framework that result in the desired lower bound on the information density. △ Less

Submitted 20 February, 2024; originally announced February 2024.

MSC Class: 94A17 ACM Class: H.1.1

Coordination Coding with Causal Encoder for Vector-valued Witsenhausen Counterexample

Authors: Mengyuan Zhao, Maël Le Treust, Tobias J. Oechtering

Abstract: We investigate the Witsenhausen counterexample in a continuous vector-valued context with a causal encoder and noncausal decoder. Our main result is the optimal single-letter condition that characterizes the set of achievable Witsenhausen power costs and estimation costs, leveraging a modified weak typicality approach. In particular, we accommodate our power analysis to the causal encoder constrai… ▽ More We investigate the Witsenhausen counterexample in a continuous vector-valued context with a causal encoder and noncausal decoder. Our main result is the optimal single-letter condition that characterizes the set of achievable Witsenhausen power costs and estimation costs, leveraging a modified weak typicality approach. In particular, we accommodate our power analysis to the causal encoder constraint, and provide an improved distortion error analysis for the challenging estimation of the interim state. Interestingly, the idea of dual role of control is explicitly captured by the two auxiliary random variables. △ Less

Submitted 30 January, 2024; originally announced January 2024.

Comments: This work has been submitted to 2024 IEEE International Symposium on Information Theory (ISIT 2024)

Private Variable-Length Coding with Zero Leakage

Authors: Amirreza Zamani, Tobias J. Oechtering, Deniz Gündüz, Mikael Skoglund

Abstract: A private compression design problem is studied, where an encoder observes useful data $Y$, wishes to compress it using variable length code and communicates it through an unsecured channel. Since $Y$ is correlated with private attribute $X$, the encoder uses a private compression mechanism to design encoded message $\cal C$ and sends it over the channel. An adversary is assumed to have access to… ▽ More A private compression design problem is studied, where an encoder observes useful data $Y$, wishes to compress it using variable length code and communicates it through an unsecured channel. Since $Y$ is correlated with private attribute $X$, the encoder uses a private compression mechanism to design encoded message $\cal C$ and sends it over the channel. An adversary is assumed to have access to the output of the encoder, i.e., $\cal C$, and tries to estimate $X$. Furthermore, it is assumed that both encoder and decoder have access to a shared secret key $W$. The design goal is to encode message $\cal C$ with minimum possible average length that satisfies a perfect privacy constraint. To do so we first consider two different privacy mechanism design problems and find upper bounds on the entropy of the optimizers by solving a linear program. We use the obtained optimizers to design $\cal C$. In two cases we strengthen the existing bounds: 1. $|\mathcal{X}|\geq |\mathcal{Y}|$; 2. The realization of $(X,Y)$ follows a specific joint distribution. In particular, considering the second case we use two-part construction coding to achieve the upper bounds. Furthermore, in a numerical example we study the obtained bounds and show that they can improve the existing results. △ Less

Submitted 29 October, 2023; originally announced October 2023.

Comments: arXiv admin note: text overlap with arXiv:2306.13184, arXiv:2309.09034, arXiv:2211.15525, arXiv:2310.19122

Journal ref: 2023 IEEE International Workshop on Information Forensics and Security (WIFS)

Private Variable-Length Coding with Non-zero Leakage

Authors: Amirreza Zamani, Tobias J. Oechtering, Mikael Skoglund

Abstract: A private compression design problem is studied, where an encoder observes useful data $Y$, wishes to compress it using variable length code and communicates it through an unsecured channel. Since $Y$ is correlated with private data $X$, the encoder uses a private compression mechanism to design encoded message $\cal C$ and sends it over the channel. An adversary is assumed to have access to the o… ▽ More A private compression design problem is studied, where an encoder observes useful data $Y$, wishes to compress it using variable length code and communicates it through an unsecured channel. Since $Y$ is correlated with private data $X$, the encoder uses a private compression mechanism to design encoded message $\cal C$ and sends it over the channel. An adversary is assumed to have access to the output of the encoder, i.e., $\cal C$, and tries to estimate $X$. Furthermore, it is assumed that both encoder and decoder have access to a shared secret key $W$. In this work, we generalize the perfect privacy (secrecy) assumption and consider a non-zero leakage between the private data $X$ and encoded message $\cal C$. The design goal is to encode message $\cal C$ with minimum possible average length that satisfies non-perfect privacy constraints. We find upper and lower bounds on the average length of the encoded message using different privacy metrics and study them in special cases. For the achievability we use two-part construction coding and extended versions of Functional Representation Lemma. Lastly, in an example we show that the bounds can be asymptotically tight. △ Less

Submitted 20 November, 2023; v1 submitted 29 October, 2023; originally announced October 2023.

Comments: arXiv admin note: text overlap with arXiv:2306.13184

Private Variable-Length Coding with Sequential Encoder

Authors: Amirreza Zamani, Tobias J. Oechtering, Deniz Gündüz, Mikael Skoglund

Abstract: A multi-user private data compression problem is studied. A server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is connected to an encoder. The encoder is connected through an unsecured link to a user. We assume that each file $Y_i$ is arbitrarily correlated with a private attribute $X$, which is assumed to be accessible by the encoder. Moreover, an adversary i… ▽ More A multi-user private data compression problem is studied. A server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is connected to an encoder. The encoder is connected through an unsecured link to a user. We assume that each file $Y_i$ is arbitrarily correlated with a private attribute $X$, which is assumed to be accessible by the encoder. Moreover, an adversary is assumed to have access to the link. The users and the encoder have access to a shared secret key $W$. We assume that at each time the user asks for a file $Y_{d_i}$, where $(d_1,\ldots,d_K)$ corresponds to the demand vector. The goal is to design the delivered message $\mathcal {C}=(\mathcal {C}_1,\ldots,\mathcal {C}_K)$ after the user send his demands to the encoder such that the average length of $\mathcal{C}$ is minimized, while satisfying: i. The message $\cal C$ does not reveal any information about $X$, i.e., $X$ and $\mathcal{C}$ are independent, which corresponds to the perfect privacy constraint; ii. The user is able to decode its demands, $Y_{d_i}$, by using $\cal C$, and the shared key $W$. Here, the encoder sequentially encode each demand $Y_{d_i}$ at time $i$, using the shared key and previous encoded messages. We propose a variable-length coding scheme that uses privacy-aware compression techniques. We study proposed upper and lower bounds on the average length of $\mathcal{C}$ in an example. Finally, we study an application considering cache-aided networks. △ Less

Submitted 16 September, 2023; originally announced September 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2306.13184

New Privacy Mechanism Design With Direct Access to the Private Data

Authors: Amirreza Zamani, Tobias J. Oechtering, Mikael Skoglund

Abstract: The design of a statistical signal processing privacy problem is studied where the private data is assumed to be observable. In this work, an agent observes useful data $Y$, which is correlated with private data $X$, and wants to disclose the useful information to a user. A statistical privacy mechanism is employed to generate data $U$ based on $(X,Y)$ that maximizes the revealed information about… ▽ More The design of a statistical signal processing privacy problem is studied where the private data is assumed to be observable. In this work, an agent observes useful data $Y$, which is correlated with private data $X$, and wants to disclose the useful information to a user. A statistical privacy mechanism is employed to generate data $U$ based on $(X,Y)$ that maximizes the revealed information about $Y$ while satisfying a privacy criterion. To this end, we use extended versions of the Functional Representation Lemma and Strong Functional Representation Lemma and combine them with a simple observation which we call separation technique. New lower bounds on privacy-utility trade-off are derived and we show that they can improve the previous bounds. We study the obtained bounds in different scenarios and compare them with previous results. △ Less

Submitted 16 September, 2023; originally announced September 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2201.08738, arXiv:2212.12475

Cache-Aided Private Variable-Length Coding with Zero and Non-Zero Leakage

Authors: Amirreza Zamani, Tobias J. Oechtering, Deniz Gündüz, Mikael Skoglund

Abstract: A private cache-aided compression problem is studied, where a server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is connected through a shared link to $K$ users, each equipped with a local cache of size $MF$ bits. In the placement phase, the server fills the users$'$ caches without knowing their demands, while the delivery phase takes place after the users sen… ▽ More A private cache-aided compression problem is studied, where a server has access to a database of $N$ files, $(Y_1,...,Y_N)$, each of size $F$ bits and is connected through a shared link to $K$ users, each equipped with a local cache of size $MF$ bits. In the placement phase, the server fills the users$'$ caches without knowing their demands, while the delivery phase takes place after the users send their demands to the server. We assume that each file $Y_i$ is arbitrarily correlated with a private attribute $X$, and an adversary is assumed to have access to the shared link. The users and the server have access to a shared key $W$. The goal is to design the cache contents and the delivered message $\cal C$ such that the average length of $\mathcal{C}$ is minimized, while satisfying: i. The response $\cal C$ does not reveal any information about $X$, i.e., $X$ and $\cal C$ are independent, which corresponds to the perfect privacy constraint; ii. User $i$ is able to decode its demand, $Y_{d_i}$, by using $\cal C$, its local cache $Z_i$, and the shared key $W$. Since the database is correlated with $X$, existing codes for cache-aided delivery do not satisfy the perfect privacy condition. Indeed, we propose a variable-length coding scheme that combines privacy-aware compression with coded caching techniques. In particular, we use two-part code construction and Functional Representation Lemma. Finally, we extend the results to the case, where $X$ and $\mathcal{C}$ can be correlated, i.e., non-zero leakage is allowed. △ Less

Submitted 22 June, 2023; originally announced June 2023.

Secure Block Joint Source-Channel Coding with Sequential Encoding

Authors: Hamid Ghourchian, Tobias J. Oechtering, Mikael Skoglund

Abstract: We extend the results of Ghourchian et al. [IEEE JSAIT-2021], to joint source-channel coding with eavesdrop**. Our work characterizes the sequential encoding process using the cumulative rate distribution functions (CRDF) and includes a security constraint using the cumulative leakage distribution functions (CLF). The information leakage is defined based on the mutual information between the sou… ▽ More We extend the results of Ghourchian et al. [IEEE JSAIT-2021], to joint source-channel coding with eavesdrop**. Our work characterizes the sequential encoding process using the cumulative rate distribution functions (CRDF) and includes a security constraint using the cumulative leakage distribution functions (CLF). The information leakage is defined based on the mutual information between the source and the output of the wiretap channel to the eavesdropper. We derive inner and outer bounds on the achievable CRDF for a given source and CLF, and show that the bounds are tight when the distribution achieving the capacity of the wiretap channel is the same as the one achieving the capacity of the channel. △ Less

Submitted 10 May, 2023; originally announced May 2023.

Comments: Long version of the accepted paper in the International Symposium on Information Theory (ISIT 2023). 23 pages, 1 figure. arXiv admin note: text overlap with arXiv:2108.13295

Thompson Sampling Regret Bounds for Contextual Bandits with sub-Gaussian rewards

Authors: Amaury Gouverneur, Borja Rodríguez-Gálvez, Tobias J. Oechtering, Mikael Skoglund

Abstract: In this work, we study the performance of the Thompson Sampling algorithm for Contextual Bandit problems based on the framework introduced by Neu et al. and their concept of lifted information ratio. First, we prove a comprehensive bound on the Thompson Sampling expected cumulative regret that depends on the mutual information of the environment parameters and the history. Then, we introduce new b… ▽ More In this work, we study the performance of the Thompson Sampling algorithm for Contextual Bandit problems based on the framework introduced by Neu et al. and their concept of lifted information ratio. First, we prove a comprehensive bound on the Thompson Sampling expected cumulative regret that depends on the mutual information of the environment parameters and the history. Then, we introduce new bounds on the lifted information ratio that hold for sub-Gaussian rewards, thus generalizing the results from Neu et al. which analysis requires binary rewards. Finally, we provide explicit regret bounds for the special cases of unstructured bounded contextual bandits, structured bounded contextual bandits with Laplace likelihood, structured Bernoulli bandits, and bounded linear contextual bandits. △ Less

Submitted 26 April, 2023; originally announced April 2023.

Comments: 8 pages: 5 of the main text, 1 of references, and 2 of appendices. Accepted to ISIT 2023

Pointwise Maximal Leakage on General Alphabets

Authors: Sara Saeidian, Giulia Cervia, Tobias J. Oechtering, Mikael Skoglund

Abstract: Pointwise maximal leakage (PML) is an operationally meaningful privacy measure that quantifies the amount of information leaking about a secret $X$ to a single outcome of a related random variable $Y$. In this paper, we extend the notion of PML to random variables on arbitrary probability spaces. We develop two new definitions: First, we extend PML to countably infinite random variables by conside… ▽ More Pointwise maximal leakage (PML) is an operationally meaningful privacy measure that quantifies the amount of information leaking about a secret $X$ to a single outcome of a related random variable $Y$. In this paper, we extend the notion of PML to random variables on arbitrary probability spaces. We develop two new definitions: First, we extend PML to countably infinite random variables by considering adversaries who aim to guess the value of discrete (finite or countably infinite) functions of $X$. Then, we consider adversaries who construct estimates of $X$ that maximize the expected value of their corresponding gain functions. We use this latter setup to introduce a highly versatile form of PML that captures many scenarios of practical interest whose definition requires no assumptions about the underlying probability spaces. △ Less

Submitted 16 April, 2023; originally announced April 2023.

Comments: Accepted for presentation at ISIT2023

Inferential Privacy: From Impossibility to Database Privacy

Authors: Sara Saeidian, Giulia Cervia, Tobias J. Oechtering, Mikael Skoglund

Abstract: We investigate the possibility of guaranteeing inferential privacy for mechanisms that release useful information about some data containing sensitive information, denoted by $X$. We describe a general model of utility and privacy in which utility is achieved by disclosing the value of low-entropy features of $X$, while privacy is maintained by kee** high-entropy features of $X$ secret. Adopting… ▽ More We investigate the possibility of guaranteeing inferential privacy for mechanisms that release useful information about some data containing sensitive information, denoted by $X$. We describe a general model of utility and privacy in which utility is achieved by disclosing the value of low-entropy features of $X$, while privacy is maintained by kee** high-entropy features of $X$ secret. Adopting this model, we prove that meaningful inferential privacy guarantees can be obtained, even though this is commonly considered to be impossible by the well-known result of Dwork and Naor. Then, we specifically discuss a privacy measure called pointwise maximal leakage (PML) whose guarantees are of the inferential type. We use PML to show that differential privacy admits an inferential formulation: it describes the information leaking about a single entry in a database assuming that every other entry is known, and considering the worst-case distribution on the data. Moreover, we define inferential instance privacy (IIP) as a bound on the (non-conditional) information leaking about a single entry in the database under the worst-case distribution, and show that it is equivalent to free-lunch privacy. Overall, our approach to privacy unifies, formalizes, and explains many existing ideas, e.g., why the informed adversary assumption may lead to underestimating the information leaking about each entry in the database. Furthermore, insights obtained from our results suggest general methods for improving privacy analyses; for example, we argue that smaller privacy parameters can be obtained by excluding low-entropy prior distributions from protection. △ Less

Submitted 14 March, 2023; originally announced March 2023.

On the Privacy-Utility Trade-off With and Without Direct Access to the Private Data

Authors: Amirreza Zamani, Tobias J. Oechtering, Mikael Skoglund

Abstract: We study an information theoretic privacy mechanism design problem for two scenarios where the private data is either observable or hidden. In each scenario, we first consider bounded mutual information as privacy leakage criterion, then we use two different per-letter privacy constraints. In the first scenario, an agent observes useful data $Y$ that is correlated with private data $X$, and wishes… ▽ More We study an information theoretic privacy mechanism design problem for two scenarios where the private data is either observable or hidden. In each scenario, we first consider bounded mutual information as privacy leakage criterion, then we use two different per-letter privacy constraints. In the first scenario, an agent observes useful data $Y$ that is correlated with private data $X$, and wishes to disclose the useful information to a user. A privacy mechanism is designed to generate disclosed data $U$ which maximizes the revealed information about $Y$ while satisfying a bounded privacy leakage constraint. In the second scenario, the agent has additionally access to the private data. To this end, we first extend the Functional Representation Lemma and Strong Functional Representation Lemma by relaxing the independence condition and thereby allowing a certain leakage to find lower bounds for the second scenario with different privacy leakage constraints. Furthermore, upper and lower bounds are derived in the first scenario considering different privacy constraints. In particular, for the case where no leakage is allowed, our upper and lower bounds improve previous bounds. Moreover, considering bounded mutual information as privacy constraint we show that if the common information and mutual information between $X$ and $Y$ are equal, then the attained upper bound in the second scenario is tight. Finally, the privacy-utility trade-off with prioritized private data is studied where part of $X$, i.e., $X_1$, is more private than the remaining part, i.e., $X_2$, and we provide lower and upper bounds. △ Less

Submitted 23 December, 2022; originally announced December 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2205.04881, arXiv:2201.08738

Multi-User Privacy Mechanism Design with Non-zero Leakage

Authors: Amirreza Zamani, Tobias J. Oechtering, Mikael Skoglund

Abstract: A privacy mechanism design problem is studied through the lens of information theory. In this work, an agent observes useful data $Y=(Y_1,...,Y_N)$ that is correlated with private data $X=(X_1,...,X_N)$ which is assumed to be also accessible by the agent. Here, we consider $K$ users where user $i$ demands a sub-vector of $Y$, denoted by $C_{i}$. The agent wishes to disclose $C_{i}$ to user $i$. Si… ▽ More A privacy mechanism design problem is studied through the lens of information theory. In this work, an agent observes useful data $Y=(Y_1,...,Y_N)$ that is correlated with private data $X=(X_1,...,X_N)$ which is assumed to be also accessible by the agent. Here, we consider $K$ users where user $i$ demands a sub-vector of $Y$, denoted by $C_{i}$. The agent wishes to disclose $C_{i}$ to user $i$. Since $C_{i}$ is correlated with $X$ it can not be disclosed directly. A privacy mechanism is designed to generate disclosed data $U$ which maximizes a linear combinations of the users utilities while satisfying a bounded privacy constraint in terms of mutual information. In a similar work it has been assumed that $X_i$ is a deterministic function of $Y_i$, however in this work we let $X_i$ and $Y_i$ be arbitrarily correlated. First, an upper bound on the privacy-utility trade-off is obtained by using a specific transformation, Functional Representation Lemma and Strong Functional Representaion Lemma, then we show that the upper bound can be decomposed into $N$ parallel problems. Next, lower bounds on privacy-utility trade-off are derived using Functional Representation Lemma and Strong Functional Representaion Lemma. The upper bound is tight within a constant and the lower bounds assert that the disclosed data is independent of all $\{X_j\}_{i=1}^N$ except one which we allocate the maximum allowed leakage to it. Finally, the obtained bounds are studied in special cases. △ Less

Submitted 28 November, 2022; originally announced November 2022.

Comments: arXiv admin note: text overlap with arXiv:2205.04881, arXiv:2201.08738

An Information-Theoretic Analysis of Bayesian Reinforcement Learning

Authors: Amaury Gouverneur, Borja Rodríguez-Gálvez, Tobias J. Oechtering, Mikael Skoglund

Abstract: Building on the framework introduced by Xu and Raginksy [1] for supervised learning problems, we study the best achievable performance for model-based Bayesian reinforcement learning problems. With this purpose, we define minimum Bayesian regret (MBR) as the difference between the maximum expected cumulative reward obtainable either by learning from the collected data or by knowing the environment… ▽ More Building on the framework introduced by Xu and Raginksy [1] for supervised learning problems, we study the best achievable performance for model-based Bayesian reinforcement learning problems. With this purpose, we define minimum Bayesian regret (MBR) as the difference between the maximum expected cumulative reward obtainable either by learning from the collected data or by knowing the environment and its dynamics. We specialize this definition to reinforcement learning problems modeled as Markov decision processes (MDPs) whose kernel parameters are unknown to the agent and whose uncertainty is expressed by a prior distribution. One method for deriving upper bounds on the MBR is presented and specific bounds based on the relative entropy and the Wasserstein distance are given. We then focus on two particular cases of MDPs, the multi-armed bandit problem (MAB) and the online optimization with partial feedback problem. For the latter problem, we show that our bounds can recover from below the current information-theoretic bounds by Russo and Van Roy [2]. △ Less

Submitted 18 July, 2022; originally announced July 2022.

Comments: 10 pages: 6 of the main text, 1 of references, and 3 of appendices

Pointwise Maximal Leakage

Authors: Sara Saeidian, Giulia Cervia, Tobias J. Oechtering, Mikael Skoglund

Abstract: We introduce a privacy measure called pointwise maximal leakage, generalizing the pre-existing notion of maximal leakage, which quantifies the amount of information leaking about a secret $X$ by disclosing a single outcome of a (randomized) function calculated on $X$. Pointwise maximal leakage is a robust and operationally meaningful privacy measure that captures the largest amount of information… ▽ More We introduce a privacy measure called pointwise maximal leakage, generalizing the pre-existing notion of maximal leakage, which quantifies the amount of information leaking about a secret $X$ by disclosing a single outcome of a (randomized) function calculated on $X$. Pointwise maximal leakage is a robust and operationally meaningful privacy measure that captures the largest amount of information leaking about $X$ to adversaries seeking to guess arbitrary (possibly randomized) functions of $X$, or equivalently, aiming to maximize arbitrary gain functions. We study several properties of pointwise maximal leakage, e.g., how it composes over multiple outcomes, how it is affected by pre- and post-processing, etc. Furthermore, we propose to view information leakage as a random variable which, in turn, allows us to regard privacy guarantees as requirements imposed on different statistical properties of the information leakage random variable. We define several privacy guarantees and study how they behave under pre-processing, post-processing and composition. Finally, we examine the relationship between pointwise maximal leakage and other privacy notions such as local differential privacy, local information privacy, $f$-information, and so on. Overall, our paper constructs a robust and flexible framework for privacy risk assessment whose central notion has a strong operational meaning which can be adapted to a variety of applications and practical scenarios. △ Less

Submitted 15 August, 2023; v1 submitted 10 May, 2022; originally announced May 2022.

Comments: Results unchanged. New examples added. This version has been accepted for publication in IEEE Transactions on Information Theory

Bounds for Privacy-Utility Trade-off with Per-letter Privacy Constraints and Non-zero Leakage

Authors: Amirreza Zamani, Tobias J. Oechtering, Mikael Skoglund

Abstract: An information theoretic privacy mechanism design problem for two scenarios is studied where the private data is either hidden or observable. In each scenario, privacy leakage constraints are considered using two different measures. In these scenarios the private data is hidden or observable. In the first scenario, an agent observes useful data $Y$ that is correlated with private data $X$, and wis… ▽ More An information theoretic privacy mechanism design problem for two scenarios is studied where the private data is either hidden or observable. In each scenario, privacy leakage constraints are considered using two different measures. In these scenarios the private data is hidden or observable. In the first scenario, an agent observes useful data $Y$ that is correlated with private data $X$, and wishes to disclose the useful information to a user. A privacy mechanism is designed to generate disclosed data $U$ which maximizes the revealed information about $Y$ while satisfying a per-letter privacy constraint. In the second scenario, the agent has additionally access to the private data. First, the Functional Representation Lemma and Strong Functional Representation Lemma are extended by relaxing the independence condition to find a lower bound considering the second scenario. Next, lower bounds as well as upper bounds on privacy-utility trade-off are derived for both scenarios. In particular, for the case where $X$ is deterministic function of $Y$, we show that our upper and lower bounds are asymptotically optimal considering the first scenario. △ Less

Submitted 10 May, 2022; originally announced May 2022.

Comments: arXiv admin note: text overlap with arXiv:2201.08738

Bounds for Privacy-Utility Trade-off with Non-zero Leakage

Authors: Amirreza Zamani, Tobias J. Oechtering, Mikael Skoglund

Abstract: The design of privacy mechanisms for two scenarios is studied where the private data is hidden or observable. In the first scenario, an agent observes useful data $Y$, which is correlated with private data $X$, and wants to disclose the useful information to a user. A privacy mechanism is employed to generate data $U$ that maximizes the revealed information about $Y$ while satisfying a privacy cri… ▽ More The design of privacy mechanisms for two scenarios is studied where the private data is hidden or observable. In the first scenario, an agent observes useful data $Y$, which is correlated with private data $X$, and wants to disclose the useful information to a user. A privacy mechanism is employed to generate data $U$ that maximizes the revealed information about $Y$ while satisfying a privacy criterion. In the second scenario, the agent has additionally access to the private data. To this end, the Functional Representation Lemma and Strong Functional Representation Lemma are extended relaxing the independence condition and thereby allowing a certain leakage. Lower bounds on privacy-utility trade-off are derived for the second scenario as well as upper bounds for both scenarios. In particular, for the case where no leakage is allowed, our upper and lower bounds improve previous bounds. △ Less

Submitted 13 January, 2023; v1 submitted 21 January, 2022; originally announced January 2022.

Secure Source Coding with Side-information at Decoder and Shared Key at Encoder and Decoder

Authors: Hamid Ghourchian, Photios A. Stavrou, Tobias J. Oechtering, Mikael Skoglund

Abstract: We study the problem of rate-distortion-equivocation with side-information only available at the decoder when an independent private random key is shared between the sender and the receiver. The sender compresses the sequence, and the receiver reconstructs it such that the average distortion between the source and the output is limited. The equivocation is measured at an eavesdropper that intercep… ▽ More We study the problem of rate-distortion-equivocation with side-information only available at the decoder when an independent private random key is shared between the sender and the receiver. The sender compresses the sequence, and the receiver reconstructs it such that the average distortion between the source and the output is limited. The equivocation is measured at an eavesdropper that intercepts the source encoded message, utilizing side-information correlated with the source and the side-information at the decoder. We have derived the entire achievable rate-distortion-equivocation region for this problem. △ Less

Submitted 23 January, 2022; v1 submitted 3 September, 2021; originally announced September 2021.

Comments: Long version of the accepted paper in the Information Theory Workshop (ITW 2021). 23 pages, 4 figures

Secure Block Source Coding with Sequential Encoding

Authors: Hamid Ghourchian, Photios A. Stavrou, Tobias J. Oechtering, Mikael Skoglund

Abstract: We introduce fundamental bounds on achievable cumulative rate distribution functions (CRDF) to characterize a sequential encoding process that ensures lossless or lossy reconstruction subject to an average distortion criterion using a non-causal decoder. The CRDF describes the rate resources spent sequentially to compress the sequence. We also include a security constraint that affects the set of… ▽ More We introduce fundamental bounds on achievable cumulative rate distribution functions (CRDF) to characterize a sequential encoding process that ensures lossless or lossy reconstruction subject to an average distortion criterion using a non-causal decoder. The CRDF describes the rate resources spent sequentially to compress the sequence. We also include a security constraint that affects the set of achievable CRDF. The information leakage is defined sequentially based on the mutual information between the source and its compressed representation, as it evolves. To characterize the security constraints, we introduce the concept of cumulative leakage distribution functions (CLF), which determines the allowed information leakage as distributed over encoded sub-blocks. Utilizing tools from majorization theory, we derive necessary and sufficient conditions on the achievable CRDF for a given independent and identically distributed (IID) source and CLF. One primary result of this paper is that the concave-hull of the CRDF characterizes the optimal achievable rate distribution. △ Less

Submitted 30 August, 2021; originally announced August 2021.

Comments: 38 pages, 5 figures

Data Disclosure with Non-zero Leakage and Non-invertible Leakage Matrix

Authors: Amirreza Zamani, Tobias J. Oechtering, Mikael Skoglund

Abstract: We study a statistical signal processing privacy problem, where an agent observes useful data $Y$ and wants to reveal the information to a user. Since the useful data is correlated with the private data $X$, the agent employs a privacy mechanism to generate data $U$ that can be released. We study the privacy mechanism design that maximizes the revealed information about $Y$ while satisfying a stro… ▽ More We study a statistical signal processing privacy problem, where an agent observes useful data $Y$ and wants to reveal the information to a user. Since the useful data is correlated with the private data $X$, the agent employs a privacy mechanism to generate data $U$ that can be released. We study the privacy mechanism design that maximizes the revealed information about $Y$ while satisfying a strong $\ell_1$-privacy criterion. When a sufficiently small leakage is allowed, we show that the optimizer vectors of the privacy mechanism design problem have a specific geometry, i.e., they are perturbations of fixed vector distributions. This geometrical structure allows us to use a local approximation of the conditional entropy. By using this approximation the original optimization problem can be reduced to a linear program so that an approximate solution for privacy mechanism can be easily obtained. The main contribution of this work is to consider non-zero leakage with a non-invertible leakage matrix. In an example inspired by water mark application, we first investigate the accuracy of the approximation. Then, we employ different measures for utility and privacy leakage to compare the privacy-utility trade-off using our approach with other methods. In particular, it has been shown that by allowing small leakage, significant utility can be achieved using our method compared to the case where no leakage is allowed. △ Less

Submitted 15 July, 2021; originally announced July 2021.

Optimal Maximal Leakage-Distortion Tradeoff

Authors: Sara Saeidian, Giulia Cervia, Tobias J. Oechtering, Mikael Skoglund

Abstract: Most methods for publishing data with privacy guarantees introduce randomness into datasets which reduces the utility of the published data. In this paper, we study the privacy-utility tradeoff by taking maximal leakage as the privacy measure and the expected Hamming distortion as the utility measure. We study three different but related problems. First, we assume that the data-generating distribu… ▽ More Most methods for publishing data with privacy guarantees introduce randomness into datasets which reduces the utility of the published data. In this paper, we study the privacy-utility tradeoff by taking maximal leakage as the privacy measure and the expected Hamming distortion as the utility measure. We study three different but related problems. First, we assume that the data-generating distribution (i.e., the prior) is known, and we find the optimal privacy mechanism that achieves the smallest distortion subject to a constraint on maximal leakage. Then, we assume that the prior belongs to some set of distributions, and we formulate a min-max problem for finding the smallest distortion achievable for the worst-case prior in the set, subject to a maximal leakage constraint. Lastly, we define a partial order on privacy mechanisms based on the largest distortion they generate. Our results show that when the prior distribution is known, the optimal privacy mechanism fully discloses symbols with the largest prior probabilities, and suppresses symbols with the smallest prior probabilities. Furthermore, we show that sets of priors that contain more uniform distributions lead to larger distortion, while privacy mechanisms that distribute the privacy budget more uniformly over the symbols create smaller worst-case distortion. △ Less

Submitted 3 May, 2021; originally announced May 2021.

$(ε, n)$ Fixed-Length Strong Coordination Capacity

Authors: Giulia Cervia, Tobias J. Oechtering, Mikael Skoglund

Abstract: This paper investigates the problem of synthesizing joint distributions in the finite-length regime. For a fixed blocklength $n$ and an upper bound on the distribution approximation $ε$, we prove a capacity result for fixed-length strong coordination. It is shown analytically that the rate conditions for the fixed-length regime are lower-bounded by the mutual information that appears in the asympt… ▽ More This paper investigates the problem of synthesizing joint distributions in the finite-length regime. For a fixed blocklength $n$ and an upper bound on the distribution approximation $ε$, we prove a capacity result for fixed-length strong coordination. It is shown analytically that the rate conditions for the fixed-length regime are lower-bounded by the mutual information that appears in the asymptotical condition plus $Q^{-1} \left(ε\right) \sqrt{ V/n}$, where $V$ is the channel dispersion, and $Q^{-1}$ is the inverse of the Gaussian cumulative distribution function. △ Less

Submitted 13 May, 2021; v1 submitted 18 January, 2021; originally announced January 2021.

Quantifying Membership Privacy via Information Leakage

Authors: Sara Saeidian, Giulia Cervia, Tobias J. Oechtering, Mikael Skoglund

Abstract: Machine learning models are known to memorize the unique properties of individual data points in a training set. This memorization capability can be exploited by several types of attacks to infer information about the training data, most notably, membership inference attacks. In this paper, we propose an approach based on information leakage for guaranteeing membership privacy. Specifically, we pr… ▽ More Machine learning models are known to memorize the unique properties of individual data points in a training set. This memorization capability can be exploited by several types of attacks to infer information about the training data, most notably, membership inference attacks. In this paper, we propose an approach based on information leakage for guaranteeing membership privacy. Specifically, we propose to use a conditional form of the notion of maximal leakage to quantify the information leaking about individual data entries in a dataset, i.e., the entrywise information leakage. We apply our privacy analysis to the Private Aggregation of Teacher Ensembles (PATE) framework for privacy-preserving classification of sensitive data and prove that the entrywise information leakage of its aggregation mechanism is Schur-concave when the injected noise has a log-concave probability density. The Schur-concavity of this leakage implies that increased consensus among teachers in labeling a query reduces its associated privacy cost. Finally, we derive upper bounds on the entrywise information leakage when the aggregation mechanism uses Laplace distributed noise. △ Less

Submitted 15 April, 2021; v1 submitted 12 October, 2020; originally announced October 2020.

Comments: To be published in IEEE Transactions on Information Forensics and Security

A Design Framework for Strongly $χ^2$-Private Data Disclosure

Authors: Amirreza Zamani, Tobias J. Oechtering, Mikael Skoglund

Abstract: In this paper, we study a stochastic disclosure control problem using information-theoretic methods. The useful data to be disclosed depend on private data that should be protected. Thus, we design a privacy mechanism to produce new data which maximizes the disclosed information about the useful data under a strong $χ^2$-privacy criterion. For sufficiently small leakage, the privacy mechanism desi… ▽ More In this paper, we study a stochastic disclosure control problem using information-theoretic methods. The useful data to be disclosed depend on private data that should be protected. Thus, we design a privacy mechanism to produce new data which maximizes the disclosed information about the useful data under a strong $χ^2$-privacy criterion. For sufficiently small leakage, the privacy mechanism design problem can be geometrically studied in the space of probability distributions by a local approximation of the mutual information. By using methods from Euclidean information geometry, the original highly challenging optimization problem can be reduced to a problem of finding the principal right-singular vector of a matrix, which characterizes the optimal privacy mechanism. In two extensions we first consider a scenario where an adversary receives a noisy version of the user's message and then we look for a mechanism which finds $U$ based on observing $X$, maximizing the mutual information between $U$ and $Y$ while satisfying the privacy criterion on $U$ and $Z$ under the Markov chain $(Z,Y)-X-U$. △ Less

Submitted 22 March, 2021; v1 submitted 3 September, 2020; originally announced September 2020.

Comments: 16 pages, 2 figures

Journal ref: vol. 16, pp. 2312-2325, 2021

Remote Joint Strong Coordination and Reliable Communication

Authors: Giulia Cervia, Tobias J. Oechtering, Mikael Skoglund

Abstract: We consider a three-node network, in which two agents wish to communicate over a noisy channel, while controlling the distribution observed by a third external agent. We use strong coordination to constrain the distribution, and we provide a complete characterization of the "remote strong coordination and reliable communication" region. We consider a three-node network, in which two agents wish to communicate over a noisy channel, while controlling the distribution observed by a third external agent. We use strong coordination to constrain the distribution, and we provide a complete characterization of the "remote strong coordination and reliable communication" region. △ Less

Submitted 3 September, 2020; originally announced September 2020.

Journal ref: 2020 IEEE International Symposium on Information Theory (ISIT)

Decentralized Differentially Private Segmentation with PATE

Authors: Dominik Fay, Jens Sjölund, Tobias J. Oechtering

Abstract: When it comes to preserving privacy in medical machine learning, two important considerations are (1) kee** data local to the institution and (2) avoiding inference of sensitive information from the trained model. These are often addressed using federated learning and differential privacy, respectively. However, the commonly used Federated Averaging algorithm requires a high degree of synchroniz… ▽ More When it comes to preserving privacy in medical machine learning, two important considerations are (1) kee** data local to the institution and (2) avoiding inference of sensitive information from the trained model. These are often addressed using federated learning and differential privacy, respectively. However, the commonly used Federated Averaging algorithm requires a high degree of synchronization between participating institutions. For this reason, we turn our attention to Private Aggregation of Teacher Ensembles (PATE), where all local models can be trained independently without inter-institutional communication. The purpose of this paper is thus to explore how PATE -- originally designed for classification -- can best be adapted for semantic segmentation. To this end, we build low-dimensional representations of segmentation masks which the student can obtain through low-sensitivity queries to the private aggregator. On the Brain Tumor Segmentation (BraTS 2019) dataset, an Autoencoder-based PATE variant achieves a higher Dice coefficient for the same privacy guarantee than prior work based on noisy Federated Averaging. △ Less

Submitted 9 April, 2020; originally announced April 2020.

Comments: Under review for MICCAI 2020

On the Minimum Achievable Age of Information for General Service-Time Distributions

Authors: Jaya Prakash Champati, Ramana R. Avula, Tobias J. Oechtering, James Gross

Abstract: There is a growing interest in analysing the freshness of data in networked systems. Age of Information (AoI) has emerged as a popular metric to quantify this freshness at a given destination. There has been a significant research effort in optimizing this metric in communication and networking systems under different settings. In contrast to previous works, we are interested in a fundamental ques… ▽ More There is a growing interest in analysing the freshness of data in networked systems. Age of Information (AoI) has emerged as a popular metric to quantify this freshness at a given destination. There has been a significant research effort in optimizing this metric in communication and networking systems under different settings. In contrast to previous works, we are interested in a fundamental question, what is the minimum achievable AoI in any single-server-single-source queuing system for a given service-time distribution? To address this question, we study a problem of optimizing AoI under service preemptions. Our main result is on the characterization of the minimum achievable average peak AoI (PAoI). We obtain this result by showing that a fixed-threshold policy is optimal in the set of all randomized-threshold causal policies. We use the characterization to provide necessary and sufficient conditions for the service-time distributions under which preemptions are beneficial. △ Less

Submitted 19 January, 2020; originally announced January 2020.

Privacy-Utility Management of Hypothesis Tests

Authors: Zuxing Li, Tobias J. Oechtering

Abstract: The trade-off of hypothesis tests on the correlated privacy hypothesis and utility hypothesis is studied. The error exponent of the Bayesian composite hypothesis test on the privacy or utility hypothesis can be characterized by the corresponding minimal Chernoff information rate. An optimal management protects the privacy by minimizing the error exponent of the privacy hypothesis test and meanwhil… ▽ More The trade-off of hypothesis tests on the correlated privacy hypothesis and utility hypothesis is studied. The error exponent of the Bayesian composite hypothesis test on the privacy or utility hypothesis can be characterized by the corresponding minimal Chernoff information rate. An optimal management protects the privacy by minimizing the error exponent of the privacy hypothesis test and meanwhile guarantees the utility hypothesis testing performance by satisfying a lower bound on the corresponding minimal Chernoff information rate. The asymptotic minimum error exponent of the privacy hypothesis test is shown to be characterized by the infimum of corresponding minimal Chernoff information rates subject to the utility guarantees. △ Less

Submitted 12 September, 2018; originally announced September 2018.

Comments: accepted in IEEE Information Theory Workshop 2018

Privacy against a Hypothesis Testing Adversary

Authors: Zuxing Li, Tobias J. Oechtering, Deniz Gunduz

Abstract: Privacy against an adversary (AD) that tries to detect the underlying privacy-sensitive data distribution is studied. The original data sequence is assumed to come from one of the two known distributions, and the privacy leakage is measured by the probability of error of the binary hypothesis test carried out by the AD. A management unit (MU) is allowed to manipulate the original data sequence in… ▽ More Privacy against an adversary (AD) that tries to detect the underlying privacy-sensitive data distribution is studied. The original data sequence is assumed to come from one of the two known distributions, and the privacy leakage is measured by the probability of error of the binary hypothesis test carried out by the AD. A management unit (MU) is allowed to manipulate the original data sequence in an online fashion, while satisfying an average distortion constraint. The goal of the MU is to maximize the minimal type II probability of error subject to a constraint on the type I probability of error assuming an adversarial Neyman-Pearson test, or to maximize the minimal error probability assuming an adversarial Bayesian test. The asymptotic exponents of the maximum minimal type II probability of error and the maximum minimal error probability are shown to be characterized by a Kullback-Leibler divergence rate and a Chernoff information rate, respectively. Privacy performances of particular management policies, the memoryless hypothesis-aware policy and the hypothesis-unaware policy with memory, are compared. The proposed formulation can also model adversarial example generation with minimal data manipulation to fool classifiers. Lastly, the results are applied to a smart meter privacy problem, where the user's energy consumption is manipulated by adaptively using a renewable energy source in order to hide user's activity from the energy provider. △ Less

Submitted 9 March, 2019; v1 submitted 5 July, 2018; originally announced July 2018.

Comments: published by IEEE Transactions on Information Forensics and Security

Journal ref: IEEE Transactions on Information Forensics and Security, vol. 14, no. 6, pp. 1567-1581, 2019

Privacy-preserving smart meter control strategy including energy storage losses

Authors: Ramana R. Avula, Tobias J. Oechtering, Daniel Månsson

Abstract: Privacy-preserving smart meter control strategies proposed in the literature so far make some ideal assumptions such as instantaneous control without delay, lossless energy storage systems etc. In this paper, we present a one-step-ahead predictive control strategy using Bayesian risk to measure and control privacy leakage with an energy storage system. The controller estimates energy state using a… ▽ More Privacy-preserving smart meter control strategies proposed in the literature so far make some ideal assumptions such as instantaneous control without delay, lossless energy storage systems etc. In this paper, we present a one-step-ahead predictive control strategy using Bayesian risk to measure and control privacy leakage with an energy storage system. The controller estimates energy state using a three-circuit energy storage model to account for steady-state energy losses. With numerical experiments, the controller is evaluated with real household consumption data using a state-of-the-art adversarial algorithm. Results show that the state estimation of the energy storage system significantly affects the controller's performance. The results also show that the privacy leakage can be effectively reduced using an energy storage system but at the expense of energy loss. △ Less

Submitted 11 July, 2018; v1 submitted 21 March, 2018; originally announced March 2018.

Comments: 6 pages

Secure Estimation and Zero-Error Secrecy Capacity

Authors: Moritz Wiese, Tobias J. Oechtering, Karl Henrik Johansson, Panos Papadimitratos, Henrik Sandberg, Mikael Skoglund

Abstract: We study the problem of securely estimating the states of an unstable dynamical system subject to nonstochastic disturbances. The estimator obtains all its information through an uncertain channel which is subject to nonstochastic disturbances as well, and an eavesdropper obtains a disturbed version of the channel inputs through a second uncertain channel. An encoder observes and block-encodes the… ▽ More We study the problem of securely estimating the states of an unstable dynamical system subject to nonstochastic disturbances. The estimator obtains all its information through an uncertain channel which is subject to nonstochastic disturbances as well, and an eavesdropper obtains a disturbed version of the channel inputs through a second uncertain channel. An encoder observes and block-encodes the states in such a way that, upon sending the generated codeword, the estimator's error is bounded and such that a security criterion is satisfied ensuring that the eavesdropper obtains as little state information as possible. Two security criteria are considered and discussed with the help of a numerical example. A sufficient condition on the uncertain wiretap channel, i.e., the pair formed by the uncertain channel from encoder to estimator and the uncertain channel from encoder to eavesdropper, is derived which ensures that a bounded estimation error and security are achieved. This condition is also shown to be necessary for a subclass of uncertain wiretap channels. To formulate the condition, the zero-error secrecy capacity of uncertain wiretap channels is introduced, i.e., the maximal rate at which data can be transmitted from the encoder to the estimator in such a way that the eavesdropper is unable to reconstruct the transmitted data. Lastly, the zero-error secrecy capacity of uncertain wiretap channels is studied. △ Less

Submitted 14 July, 2017; v1 submitted 16 December, 2016; originally announced December 2016.

Comments: Submitted to IEEE Transactions on Automatic Control

Massive MIMO Pilot Retransmission Strategies for Robustification against Jamming

Authors: Tan Tai Do, Hien Quoc Ngo, Trung Q. Duong, Tobias J. Oechtering, Mikael Skoglund

Abstract: This letter proposes anti-jamming strategies based on pilot retransmission for a single user uplink massive MIMO under jamming attack. A jammer is assumed to attack the system both in the training and data transmission phases. We first derive an achievable rate which enables us to analyze the effect of jamming attacks on the system performance. Counter-attack strategies are then proposed to mitiga… ▽ More This letter proposes anti-jamming strategies based on pilot retransmission for a single user uplink massive MIMO under jamming attack. A jammer is assumed to attack the system both in the training and data transmission phases. We first derive an achievable rate which enables us to analyze the effect of jamming attacks on the system performance. Counter-attack strategies are then proposed to mitigate this effect under two different scenarios: random and deterministic jamming attacks. Numerical results illustrate our analysis and benefit of the proposed schemes. △ Less

Submitted 24 November, 2016; originally announced November 2016.

Comments: Accepted for publication in the IEEE Wireless Communications Letters

Uncertain Wiretap Channels and Secure Estimation

Authors: Moritz Wiese, Karl Henrik Johansson, Tobias J. Oechtering, Panos Papadimitratos, Henrik Sandberg, Mikael Skoglund

Abstract: Uncertain wiretap channels are introduced. Their zero-error secrecy capacity is defined. If the sensor-estimator channel is perfect, it is also calculated. Further properties are discussed. The problem of estimating a dynamical system with nonstochastic disturbances is studied where the sensor is connected to the estimator and an eavesdropper via an uncertain wiretap channel. The estimator should… ▽ More Uncertain wiretap channels are introduced. Their zero-error secrecy capacity is defined. If the sensor-estimator channel is perfect, it is also calculated. Further properties are discussed. The problem of estimating a dynamical system with nonstochastic disturbances is studied where the sensor is connected to the estimator and an eavesdropper via an uncertain wiretap channel. The estimator should obtain a uniformly bounded estimation error whereas the eavesdropper's error should tend to infinity. It is proved that the system can be estimated securely if the zero-error capacity of the sensor-estimator channel is strictly larger than the logarithm of the system's unstable pole and the zero-error secrecy capacity of the uncertain wiretap channel is positive. △ Less

Submitted 1 May, 2016; originally announced May 2016.

Comments: Long version of a paper to be presented at ISIT 2016

Authentication With a Guessing Adversary

Authors: Farshad Naghibi, Tobias J. Oechtering, Mikael Skoglund

Abstract: In this paper, we consider the authentication problem where a candidate measurement presented by an unidentified user is compared to a previously stored measurement of the legitimate user, the enrollment, with respect to a certain distortion criteria for authentication. An adversary wishes to impersonate the legitimate user by guessing the enrollment until the system authenticates him. For this se… ▽ More In this paper, we consider the authentication problem where a candidate measurement presented by an unidentified user is compared to a previously stored measurement of the legitimate user, the enrollment, with respect to a certain distortion criteria for authentication. An adversary wishes to impersonate the legitimate user by guessing the enrollment until the system authenticates him. For this setting, we study the minimum number of required guesses (on average) by the adversary for a successful impersonation attack and find the complete characterization of the asymptotic exponent of this metric, referred to as the deception exponent. Our result is a direct application of the results of the Guessing problem by Arikan and Merhav [19]. Paralleling the work in [19] we also extend this result to the case where the adversary may have access to additional side information correlated to the enrollment data. The paper is a revised version of a submission to IEEE WIFS 2015, with the referencing to the paper [19] clarified compared with the conference version. △ Less

Submitted 25 November, 2015; originally announced November 2015.

Comments: 6 pages, 1 figure, revised IEEE WIFS 2015 submission

On the Entropy Computation of Large Complex Gaussian Mixture Distributions

Authors: Su Min Kim, Tan Tai Do, Tobias J. Oechtering, Gunnar Peters

Abstract: The entropy computation of Gaussian mixture distributions with a large number of components has a prohibitive computational complexity. In this paper, we propose a novel approach exploiting the sphere decoding concept to bound and approximate such entropy terms with reduced complexity and good accuracy. Moreover, we propose an SNR region based enhancement of the approximation method to reduce the… ▽ More The entropy computation of Gaussian mixture distributions with a large number of components has a prohibitive computational complexity. In this paper, we propose a novel approach exploiting the sphere decoding concept to bound and approximate such entropy terms with reduced complexity and good accuracy. Moreover, we propose an SNR region based enhancement of the approximation method to reduce the complexity even further. Using Monte-Carlo simulations, the proposed methods are numerically demonstrated for the computation of the mutual information including the entropy term of various channels with finite constellation modulations such as binary and quadratic amplitude modulation (QAM) inputs for communication applications. △ Less

Submitted 27 May, 2015; v1 submitted 12 February, 2015; originally announced February 2015.

Comments: 14 pages, Accepted to IEEE Transactions on Signal Processing

Lossy Source Coding with Reconstruction Privacy

Authors: Kittipong Kittichokechai, Tobias J. Oechtering, Mikael Skoglund

Abstract: We consider the problem of lossy source coding with side information under a privacy constraint that the reconstruction sequence at a decoder should be kept secret to a certain extent from another terminal such as an eavesdropper, a sender, or a helper. We are interested in how the reconstruction privacy constraint at a particular terminal affects the rate-distortion tradeoff. In this work, we all… ▽ More We consider the problem of lossy source coding with side information under a privacy constraint that the reconstruction sequence at a decoder should be kept secret to a certain extent from another terminal such as an eavesdropper, a sender, or a helper. We are interested in how the reconstruction privacy constraint at a particular terminal affects the rate-distortion tradeoff. In this work, we allow the decoder to use a random map**, and give inner and outer bounds to the rate-distortion-equivocation region for different cases where the side information is available non-causally and causally at the decoder. In the special case where each reconstruction symbol depends only on the source description and current side information symbol, the complete rate-distortion-equivocation region is provided. A binary example illustrating a new tradeoff due to the new privacy constraint, and a gain from the use of a stochastic decoder is given. △ Less

Submitted 24 June, 2014; v1 submitted 18 February, 2014; originally announced February 2014.

Comments: 22 pages, added proofs, to be presented at ISIT 2014

Stabilization of Linear Systems Over Gaussian Networks

Authors: Ali A. Zaidi, Tobias J. Oechtering, Serdar Yuksel, Mikael Skoglund

Abstract: The problem of remotely stabilizing a noisy linear time invariant plant over a Gaussian relay network is addressed. The network is comprised of a sensor node, a group of relay nodes and a remote controller. The sensor and the relay nodes operate subject to an average transmit power constraint and they can cooperate to communicate the observations of the plant's state to the remote controller. The… ▽ More The problem of remotely stabilizing a noisy linear time invariant plant over a Gaussian relay network is addressed. The network is comprised of a sensor node, a group of relay nodes and a remote controller. The sensor and the relay nodes operate subject to an average transmit power constraint and they can cooperate to communicate the observations of the plant's state to the remote controller. The communication links between all nodes are modeled as Gaussian channels. Necessary as well as sufficient conditions for mean-square stabilization over various network topologies are derived. The sufficient conditions are in general obtained using delay-free linear policies and the necessary conditions are obtained using information theoretic tools. Different settings where linear policies are optimal, asymptotically optimal (in certain parameters of the system) and suboptimal have been identified. For the case with noisy multi-dimensional sources controlled over scalar channels, it is shown that linear time varying policies lead to minimum capacity requirements, meeting the fundamental lower bound. For the case with noiseless sources and parallel channels, non-linear policies which meet the lower bound have been identified. △ Less

Submitted 29 July, 2013; originally announced July 2013.

Secure Source Coding with a Public Helper

Authors: Kittipong Kittichokechai, Yeow-Khiang Chia, Tobias J. Oechtering, Mikael Skoglund, Tsachy Weissman

Abstract: We consider secure multi-terminal source coding problems in the presence of a public helper. Two main scenarios are studied: 1) source coding with a helper where the coded side information from the helper is eavesdropped by an external eavesdropper; 2) triangular source coding with a helper where the helper is considered as a public terminal. We are interested in how the helper can support the sou… ▽ More We consider secure multi-terminal source coding problems in the presence of a public helper. Two main scenarios are studied: 1) source coding with a helper where the coded side information from the helper is eavesdropped by an external eavesdropper; 2) triangular source coding with a helper where the helper is considered as a public terminal. We are interested in how the helper can support the source transmission subject to a constraint on the amount of information leaked due to its public nature. We characterize the tradeoff between transmission rate, incurred distortion, and information leakage rate at the helper/eavesdropper in the form of a rate-distortion-leakage region for various classes of problems. △ Less

Submitted 4 July, 2013; originally announced July 2013.

Comments: 45 pages, 12 figures, a short version to be presented at ISIT 2013

Source Coding Problems with Conditionally Less Noisy Side Information

Authors: Roy Timo, Tobias J. Oechtering, Michèle Wigger

Abstract: A computable expression for the rate-distortion (RD) function proposed by Heegard and Berger has eluded information theory for nearly three decades. Heegard and Berger's single-letter achievability bound is well known to be optimal for \emph{physically degraded} side information; however, it is not known whether the bound is optimal for arbitrarily correlated side information (general discrete mem… ▽ More A computable expression for the rate-distortion (RD) function proposed by Heegard and Berger has eluded information theory for nearly three decades. Heegard and Berger's single-letter achievability bound is well known to be optimal for \emph{physically degraded} side information; however, it is not known whether the bound is optimal for arbitrarily correlated side information (general discrete memoryless sources). In this paper, we consider a new setup in which the side information at one receiver is \emph{conditionally less noisy} than the side information at the other. The new setup includes degraded side information as a special case, and it is motivated by the literature on degraded and less noisy broadcast channels. Our key contribution is a converse proving the optimality of Heegard and Berger's achievability bound in a new setting. The converse rests upon a certain \emph{single-letterization} lemma, which we prove using an information theoretic telesco** identity {recently presented by Kramer}. We also generalise the above ideas to two different successive-refinement problems. △ Less

Submitted 11 December, 2012; originally announced December 2012.

Iterative Source-Channel Coding Approach to Witsenhausen's Counterexample

Authors: Johannes Kron, Ather Gattami, Tobias J. Oechtering, Mikael Skoglund

Abstract: In 1968, Witsenhausen introduced his famous counterexample where he showed that even in the simple linear quadratic static team decision problem, complex nonlinear decisions could outperform any given linear decision. This problem has served as a benchmark problem for decades where researchers try to achieve the optimal solution. This paper introduces a systematic iterative source--channel coding… ▽ More In 1968, Witsenhausen introduced his famous counterexample where he showed that even in the simple linear quadratic static team decision problem, complex nonlinear decisions could outperform any given linear decision. This problem has served as a benchmark problem for decades where researchers try to achieve the optimal solution. This paper introduces a systematic iterative source--channel coding approach to solve problems of the Witsenhausen Counterexample-character. The advantage of the presented approach is its simplicity. Also, no assumptions are made about the shape of the space of policies. The minimal cost obtained using the introduced method is 0.16692462, which is the lowest known to date. △ Less

Submitted 21 May, 2012; originally announced May 2012.

Coding With Action-dependent Side Information and Additional Reconstruction Requirements

Authors: Kittipong Kittichokechai, Tobias J. Oechtering, Mikael Skoglund

Abstract: Constrained lossy source coding and channel coding with side information problems which extend the classic Wyner-Ziv and Gel'fand-Pinsker problems are considered. Inspired by applications in sensor networking and control, we first consider lossy source coding with two-sided partial side information where the quality/availability of the side information can be influenced by a cost-constrained actio… ▽ More Constrained lossy source coding and channel coding with side information problems which extend the classic Wyner-Ziv and Gel'fand-Pinsker problems are considered. Inspired by applications in sensor networking and control, we first consider lossy source coding with two-sided partial side information where the quality/availability of the side information can be influenced by a cost-constrained action sequence. A decoder reconstructs a source sequence subject to the distortion constraint, and at the same time, an encoder is additionally required to be able to estimate the decoder's reconstruction. Next, we consider the channel coding "dual" where the channel state is assumed to depend on the action sequence, and the decoder is required to decode both the transmitted message and channel input reliably. Implications on the fundamental limits of communication in discrete memoryless systems due to the additional reconstruction constraints are investigated. Single-letter expressions for the rate-distortion-cost function and channel capacity for the respective source and channel coding problems are derived. The dual relation between the two problems is discussed. Additionally, based on the two-stage coding structure and the additional reconstruction constraint of the channel coding problem, we discuss and give an interpretation of the two-stage coding condition which appears in the channel capacity expression. Besides the rate constraint on the message, this condition is a necessary and sufficient condition for reliable transmission of the channel input sequence over the channel in our "two-stage" communication problem. It is also shown in one example that there exists a case where the two-stage coding condition can be active in computing the capacity, and it thus can actively restrict the set of capacity achieving input distributions. △ Less

Submitted 7 February, 2012; originally announced February 2012.

Comments: submitted to IEEE Trans. Information Theory

Capacity of Gaussian MIMO Bidirectional Broadcast Channels

Authors: Rafael F. Wyrembelski, Tobias J. Oechtering, Igor Bjelakovic, Clemens Schnurr, Holger Boche

Abstract: We consider the broadcast phase of a three-node network, where a relay node establishes a bidirectional communication between two nodes using a spectrally efficient two-phase decode-and-forward protocol. In the first phase the two nodes transmit their messages to the relay node. Then the relay node decodes the messages and broadcasts a re-encoded composition of them in the second phase. We consi… ▽ More We consider the broadcast phase of a three-node network, where a relay node establishes a bidirectional communication between two nodes using a spectrally efficient two-phase decode-and-forward protocol. In the first phase the two nodes transmit their messages to the relay node. Then the relay node decodes the messages and broadcasts a re-encoded composition of them in the second phase. We consider Gaussian MIMO channels and determine the capacity region for the second phase which we call the Gaussian MIMO bidirectional broadcast channel. △ Less

Submitted 30 April, 2008; v1 submitted 26 March, 2008; originally announced March 2008.

Comments: Proc. IEEE International Symposium on Information Theory (ISIT 2008), Toronto, Canada, July 2008

Broadcast Capacity Region of Two-Phase Bidirectional Relaying

Authors: Tobias J. Oechtering, Igor Bjelakovic, Clemens Schnurr, Holger Boche

Abstract: In a three-node network a half-duplex relay node enables bidirectional communication between two nodes with a spectral efficient two phase protocol. In the first phase, two nodes transmit their message to the relay node, which decodes the messages and broadcast a re-encoded composition in the second phase. In this work we determine the capacity region of the broadcast phase. In this scenario eac… ▽ More In a three-node network a half-duplex relay node enables bidirectional communication between two nodes with a spectral efficient two phase protocol. In the first phase, two nodes transmit their message to the relay node, which decodes the messages and broadcast a re-encoded composition in the second phase. In this work we determine the capacity region of the broadcast phase. In this scenario each receiving node has perfect information about the message that is intended for the other node. The resulting set of achievable rates of the two-phase bidirectional relaying includes the region which can be achieved by applying XOR on the decoded messages at the relay node. We also prove the strong converse for the maximum error probability and show that this implies that the $[\eps_1,\eps_2]$-capacity region defined with respect to the average error probability is constant for small values of error parameters $\eps_1$, $\eps_2$. △ Less

Submitted 15 March, 2007; originally announced March 2007.

Comments: 25 pages, 2 figures, submitted to IEEE Transactions on Information Theory

Complete Characterization of the Equivalent MIMO Channel for Quasi-Orthogonal Space-Time Codes

Authors: A. Sezgin, T. J. Oechtering

Abstract: Recently, a quasi-orthogonal space-time block code (QSTBC) capable of achieving a significant fraction of the outage mutual information of a multiple-input-multiple output (MIMO) wireless communication system for the case of four transmit and one receive antennas was proposed. We generalize these results to $n_T=2^n$ transmit and an arbitrary number of receive antennas $n_R$. Furthermore, we com… ▽ More Recently, a quasi-orthogonal space-time block code (QSTBC) capable of achieving a significant fraction of the outage mutual information of a multiple-input-multiple output (MIMO) wireless communication system for the case of four transmit and one receive antennas was proposed. We generalize these results to $n_T=2^n$ transmit and an arbitrary number of receive antennas $n_R$. Furthermore, we completely characterize the structure of the equivalent channel for the general case and show that for all $n_T=2^n$ and $n_R$ the eigenvectors of the equivalent channel are fixed and independent from the channel realization. Furthermore, the eigenvalues of the equivalent channel are independent identically distributed random variables each following a noncentral chi-square distribution with $4n_R$ degrees of freedom. Based on these important insights into the structure of the QSTBC, we derive an analytical lower bound for the fraction of outage probability achieved with QSTBC and show that this bound is tight for low signal-to-noise-ratios (SNR) values and also for increasing number of receive antennas. We also present an upper bound, which is tight for high SNR values and derive analytical expressions for the case of four transmit antennas. Finally, by utilizing the special structure of the QSTBC we propose a new transmit strategy, which decouples the signals transmitted from different antennas in order to detect the symbols separately with a linear ML-detector rather than joint detection, an up to now only known advantage of orthogonal space-time block codes (OSTBC). △ Less

Submitted 17 December, 2004; originally announced December 2004.

Comments: 30 pages, 4 figures

Journal ref: IEEE Transactions on Information Theory, vol. 54(7), pp. 3315-3327, July, 2008