Showing 1–40 of 40 results for author: Akyol, E

Impact of Confirmation Bias on Competitive Information Spread in Social Networks

Authors: Yanbing Mao, Emrah Akyol, Naira Hovakimyan

Abstract: This paper investigates the impact of confirmation bias on competitive information spread in the cyber-social network that comprises individuals in a social network and competitive information sources in cyber layer. We formulate the problem as a zero-sum game, which admits a unique Nash equilibrium in pure strategies. We characterize the dependence of pure Nash equilibrium on the public's innate… ▽ More This paper investigates the impact of confirmation bias on competitive information spread in the cyber-social network that comprises individuals in a social network and competitive information sources in cyber layer. We formulate the problem as a zero-sum game, which admits a unique Nash equilibrium in pure strategies. We characterize the dependence of pure Nash equilibrium on the public's innate opinions, the social network topology, as well as the parameters of confirmation bias. We uncover that confirmation bias moves the equilibrium towards the center only when the innate opinions are not neutral, and this move does not occur for the competitive information sources simultaneously. Numerical examples in the context of well-known Krackhardt's advice network are provided to demonstrate the correctness of theoretical results. △ Less

Submitted 24 August, 2020; v1 submitted 26 August, 2019; originally announced August 2019.

Comments: submitted

On Inference of Network Topology and Confirmation Bias in Cyber-Social Networks

Authors: Yanbing Mao, Emrah Akyol

Abstract: This paper studies topology inference, from agent states, of a directed cyber-social network with opinion spreading dynamics model that explicitly takes confirmation bias into account. The cyber-social network comprises a set of partially connected directed network of agents at the social level, and a set of information sources at the cyber layer. The necessary and sufficient conditions for the ex… ▽ More This paper studies topology inference, from agent states, of a directed cyber-social network with opinion spreading dynamics model that explicitly takes confirmation bias into account. The cyber-social network comprises a set of partially connected directed network of agents at the social level, and a set of information sources at the cyber layer. The necessary and sufficient conditions for the existence of exact inference solution are characterized. A method for exact inference, when it is possible, of entire network topology as well as confirmation bias model parameters is proposed for the case where the bias mentioned earlier follows a piece-wise linear model. The particular case of no confirmation bias is analyzed in detail. For the setting where the model of confirmation bias is unknown, an algorithm that approximates the network topology, building on the exact inference method, is presented. This algorithm can exactly infer the weighted communication from the neighbors to the non-followers of information sources. Numerical simulations demonstrate the effectiveness of the proposed methods for different scenarios. △ Less

Submitted 25 April, 2020; v1 submitted 26 August, 2019; originally announced August 2019.

Comments: to appear in IEEE Transactions on Signal and Information Processing over Networks

Novel Stealthy Attack and Defense Strategies for Networked Control Systems

Authors: Yanbing Mao, Hamidreza Jafarnejadsani, Pan Zhao, Emrah Akyol, Naira Hovakimyan

Abstract: This paper studies novel attack and defense strategies, based on a class of stealthy attacks, namely the zero-dynamics attack (ZDA), for multi-agent control systems. ZDA poses a formidable security challenge since its attack signal is hidden in the null-space of the state-space representation of the control system and hence it can evade conventional detection methods. An intuitive defense strategy… ▽ More This paper studies novel attack and defense strategies, based on a class of stealthy attacks, namely the zero-dynamics attack (ZDA), for multi-agent control systems. ZDA poses a formidable security challenge since its attack signal is hidden in the null-space of the state-space representation of the control system and hence it can evade conventional detection methods. An intuitive defense strategy builds on changing the aforementioned representation via switching through a set of carefully crafted topologies. In this paper, we propose realistic ZDA variations where the attacker is aware of this topology-switching strategy, and hence employs the following policies to avoid detection: (i) pause, update and resume ZDA according to the knowledge of switching topologies; (ii) cooperate with a concurrent stealthy topology attack that alters network topology at switching times, such that the original ZDA is feasible under the corrupted topology. We first systematically study the proposed ZDA variations, and then develop defense strategies against them under the realistic assumption that the defender has no knowledge of attack starting, pausing, and resuming times and the number of misbehaving agents. Particularly, we characterize conditions for detectability of the proposed ZDA variations, in terms of the network topologies to be maintained, the set of agents to be monitored, and the measurements of the monitored agents that should be extracted, while simultaneously preserving the privacy of the states of the non-monitored agents. We then propose an attack detection algorithm based on the Luenberger observer, using the characterized detectability conditions. We provide numerical simulation results to demonstrate our theoretical findings. △ Less

Submitted 25 April, 2020; v1 submitted 26 August, 2019; originally announced August 2019.

Comments: to appear in IEEE TAC

On Remote Estimation with Multiple Communication Channels

Authors: Xiaobin Gao, Emrah Akyol, Tamer Basar

Abstract: This paper considers a sequential sensor scheduling and remote estimation problem with multiple communication channels. Departing from the classical remote estimation paradigm, which involves one communication channel (noiseless or noisy), we consider here the more realistic setting of two channels with different characteristics (one is cheap but noisy, the other one is costly but noiseless). We f… ▽ More This paper considers a sequential sensor scheduling and remote estimation problem with multiple communication channels. Departing from the classical remote estimation paradigm, which involves one communication channel (noiseless or noisy), we consider here the more realistic setting of two channels with different characteristics (one is cheap but noisy, the other one is costly but noiseless). We first show, via a counter-example, that the common folklore of applying symmetric threshold-based policy, which is well known to be optimal (for unimodal state densities) in the classical remote estimation problem, can no longer be optimal in our setting. In view of that, and in order to make the problem tractable, we introduce a side channel which signals to the receiver the sign of the underlying state. We show, under some technical assumptions, that a threshold-in-threshold based communication scheduling is optimal. The impact of the results is analyzed numerically based on dynamic programming. This numerical analysis reveals some rather surprising results inheriting known properties from the single channel settings, such as not exhausting all the opportunities available for the noisy channel. △ Less

Submitted 7 April, 2018; originally announced April 2018.

Spread of Information with Confirmation Bias in Cyber-Social Networks

Authors: Yanbing Mao, Sadegh Bolouki, Emrah Akyol

Abstract: This paper provides a model to investigate information spreading over cyber-social network of agents communicating with each other. The cyber-social network considered here comprises individuals and news agencies. Each individual holds a belief represented by a scalar. Individuals receive information from news agencies that are closer to their belief, confirmation bias is explicitly incorporated i… ▽ More This paper provides a model to investigate information spreading over cyber-social network of agents communicating with each other. The cyber-social network considered here comprises individuals and news agencies. Each individual holds a belief represented by a scalar. Individuals receive information from news agencies that are closer to their belief, confirmation bias is explicitly incorporated into the model. The proposed dynamics of cyber-social networks is adopted from DeGroot-Friedkin model, where the individual's opinion update mechanism is a convex combination of his innate opinion, his neighbors' opinions at the previous time step (obtained from the social network), and the opinions passed along by news agencies from cyber layer which he follows. The characteristics of the interdependent social and cyber networks are radically different here: the social network relies on trust and hence static while the news agencies are highly dynamic since they are weighted as a function of the distance between an individual state and the state of news agency to account for confirmation bias. The conditions for convergence of the aforementioned dynamics to a unique equilibrium are characterized. The estimation and exact computation of the steady-state values under non-linear and linear state-dependent weight functions are provided. Finally, the impact of polarization in the opinions of news agencies on the public opinion evolution is numerically analyzed in the context of the well-known Krackhardt's advice network. △ Less

Submitted 16 March, 2018; originally announced March 2018.

Strategic Topology Switching for Security-Part I: Consensus & Switching Times

Authors: Yanbing Mao, Emrah Akyol, Ziang Zhang

Abstract: In this two-part paper, we consider strategic topology switching for the second-order multi-agent systems under a special class of stealthy attacks, namely the "zero-dynamics" attack (ZDA). The main mathematical tool proposed here is to strategically switch the network topology to detect a possible ZDA. However, it is not clear a priori that such a switching strategy still yields consensus in this… ▽ More In this two-part paper, we consider strategic topology switching for the second-order multi-agent systems under a special class of stealthy attacks, namely the "zero-dynamics" attack (ZDA). The main mathematical tool proposed here is to strategically switch the network topology to detect a possible ZDA. However, it is not clear a priori that such a switching strategy still yields consensus in this switched system, in the normal (un-attacked) operation mode. In Part I, we propose a strategy on the switching times that enables the topology-switching algorithm proposed in Part II to reach the second-order consensus in the absence of a ZDA. Utilizing the theory of stable switched linear systems with unstable subsystems, we characterize sufficient conditions for the dwell time of topology-switching signal to reach consensus. Building on this characterization, we then propose a decentralized time-dependent topology-switching algorithm. The proposed algorithm, used in conjunction with a simplified control protocol, achieves consensus while providing substantial advantages over other control approaches: it relies only on the relative position measurements (without any requirement for velocity measurements); and it does not impose any constraint on the magnitudes of coupling weights. We finally demonstrate our theoretical findings via the numerical simulation results. △ Less

Submitted 4 March, 2019; v1 submitted 29 November, 2017; originally announced November 2017.

Comments: working paper, 12 pages

Strategic Topology Switching for Security-Part II: Detection & Switching Topologies

Authors: Yanbing Mao, Emrah Akyol, Ziang Zhang

Abstract: This two-part paper considers strategic topology switching for security in the second-order multi-agent system. In Part II, we propose a strategy on switching topologies to detect zero-dynamics attack (ZDA), whose attack-starting time is allowed to be not the initial time. We first characterize the sufficient and necessary condition for detectability of ZDA, in terms of the network topologies to b… ▽ More This two-part paper considers strategic topology switching for security in the second-order multi-agent system. In Part II, we propose a strategy on switching topologies to detect zero-dynamics attack (ZDA), whose attack-starting time is allowed to be not the initial time. We first characterize the sufficient and necessary condition for detectability of ZDA, in terms of the network topologies to be switched to and the set of agents to be monitored. We then propose an attack detection algorithm based on the Luenberger observer, using the characterized detectability condition. Employing the strategy on switching times proposed in Part I and the strategy on switching topologies proposed here, a strategic topology-switching algorithm is derived. Its primary advantages are threefold: (i) in achieving consensus in the absence of attacks, the control protocol does not need velocity measurements and the algorithm has no constraint on the magnitudes of coupling weights; (ii) in tracking system in the absence of attacks, the Luenberger observer has no constraint on the magnitudes of observer gains and the number of monitored agents, i.e., only one monitored agent's output is sufficient; (iii) in detecting ZDA, the algorithm allows the defender to have no knowledge of the attack-starting time and the number of misbehaving agents (i.e., agents under attack). Simulations are provided to verify the effectiveness of the strategic topology-switching algorithm. △ Less

Submitted 4 March, 2019; v1 submitted 29 November, 2017; originally announced November 2017.

Comments: working paper, 12 pages, second part of a two-part paper

Strategic Communication Between Prospect Theoretic Agents over a Gaussian Test Channel

Authors: Venkata Sriram Siddhardh Nadendla, Emrah Akyol, Cedric Langbort, Tamer Başar

Abstract: In this paper, we model a Stackelberg game in a simple Gaussian test channel where a human transmitter (leader) communicates a source message to a human receiver (follower). We model human decision making using prospect theory models proposed for continuous decision spaces. Assuming that the value function is the squared distortion at both the transmitter and the receiver, we analyze the effects o… ▽ More In this paper, we model a Stackelberg game in a simple Gaussian test channel where a human transmitter (leader) communicates a source message to a human receiver (follower). We model human decision making using prospect theory models proposed for continuous decision spaces. Assuming that the value function is the squared distortion at both the transmitter and the receiver, we analyze the effects of the weight functions at both the transmitter and the receiver on optimal communication strategies, namely encoding at the transmitter and decoding at the receiver, in the Stackelberg sense. We show that the optimal strategies for the behavioral agents in the Stackelberg sense are identical to those designed for unbiased agents. At the same time, we also show that the prospect-theoretic distortions at both the transmitter and the receiver are both larger than the expected distortion, thus making behavioral agents less contended than unbiased agents. Consequently, the presence of cognitive biases increases the need for transmission power in order to achieve a given distortion at both transmitter and receiver. △ Less

Submitted 28 September, 2017; v1 submitted 14 August, 2017; originally announced August 2017.

Comments: 6 pages, 3 figures, Accepted to MILCOM-2017, Corrections made in the new version

Optimal Communication Strategies in Networked Cyber-Physical Systems with Adversarial Elements

Authors: Emrah Akyol, Kenneth Rose, Tamer Basar, Cedric Langbort

Abstract: This paper studies optimal communication and coordination strategies in cyber-physical systems for both defender and attacker within a game-theoretic framework. We model the communication network of a cyber-physical system as a sensor network which involves one single Gaussian source observed by many sensors, subject to additive independent Gaussian observation noises. The sensors communicate with… ▽ More This paper studies optimal communication and coordination strategies in cyber-physical systems for both defender and attacker within a game-theoretic framework. We model the communication network of a cyber-physical system as a sensor network which involves one single Gaussian source observed by many sensors, subject to additive independent Gaussian observation noises. The sensors communicate with the estimator over a coherent Gaussian multiple access channel. The aim of the receiver is to reconstruct the underlying source with minimum mean squared error. The scenario of interest here is one where some of the sensors are captured by the attacker and they act as the adversary (jammer): they strive to maximize distortion. The receiver (estimator) knows the captured sensors but still cannot simply ignore them due to the multiple access channel, i.e., the outputs of all sensors are summed to generate the estimator input. We show that the ability of transmitter sensors to secretly agree on a random event, that is "coordination", plays a key role in the analysis... △ Less

Submitted 27 January, 2017; originally announced January 2017.

Comments: submitted to IEEE Transactions on Signal and Information Processing over Networks, Special Issue on Distributed Signal Processing for Security and Privacy in Networked Cyber-Physical Systems

Price of Transparency in Strategic Machine Learning

Authors: Emrah Akyol, Cedric Langbort, Tamer Basar

Abstract: Based on the observation that the transparency of an algorithm comes with a cost for the algorithm designer when the users (data providers) are strategic, this paper studies the impact of strategic intent of the users on the design and performance of transparent ML algorithms. We quantitatively study the {\bf price of transparency} in the context of strategic classification algorithms, by modeling… ▽ More Based on the observation that the transparency of an algorithm comes with a cost for the algorithm designer when the users (data providers) are strategic, this paper studies the impact of strategic intent of the users on the design and performance of transparent ML algorithms. We quantitatively study the {\bf price of transparency} in the context of strategic classification algorithms, by modeling the problem as a nonzero-sum game between the users and the algorithm designer. The cost of having a transparent algorithm is measured by a quantity, named here as price of transparency which is the ratio of the designer cost at the Stackelberg equilibrium, when the algorithm is transparent (which allows users to be strategic) to that of the setting where the algorithm is not transparent. △ Less

Submitted 26 October, 2016; originally announced October 2016.

Comments: 3rd Workshop on Fairness, Accountability, and Transparency in Machine Learning

Optimal Communication Scheduling and Remote Estimation over an Additive Noise Channel

Authors: Xiaobin Gao, Emrah Akyol, Tamer Basar

Abstract: This paper considers a sequential sensor scheduling and remote estimation problem with one sensor and one estimator. The sensor makes sequential observations about the state of an underlying memoryless stochastic process and makes a decision as to whether or not to send this measurement to the estimator. The sensor and the estimator have the common objective of minimizing expected distortion in th… ▽ More This paper considers a sequential sensor scheduling and remote estimation problem with one sensor and one estimator. The sensor makes sequential observations about the state of an underlying memoryless stochastic process and makes a decision as to whether or not to send this measurement to the estimator. The sensor and the estimator have the common objective of minimizing expected distortion in the estimation of the state of the process, over a finite time horizon. The sensor is either charged a cost for each transmission or constrained on transmission times. As opposed to the prior work where communication between the sensor and the estimator was assumed to be perfect (noiseless), in this work an additive noise channel with fixed power constraint is considered; hence, the sensor has to encode its message before transmission. Under some technical assumptions, we obtain the optimal encoding and estimation policies in conjunction with the optimal transmission schedule. The impact of the presence of a noisy channel is analyzed numerically based on dynamic programming. This analysis yields some rather surprising results such as a phase transition phenomenon in the number of used transmission opportunities, which was not encountered in the noiseless communication setting. △ Less

Submitted 18 October, 2016; originally announced October 2016.

Comments: Extended version of a journal submission

Hierarchical Multistage Gaussian Signaling Games in Noncooperative Communication and Control Systems

Authors: Muhammed O. Sayin, Emrah Akyol, Tamer Basar

Abstract: We analyze in this paper finite horizon hierarchical signaling games between (information provider) senders and (decision maker) receivers in a dynamic environment. The underlying information evolves in time while sender and receiver interact repeatedly. Different from the classical communication (control) models, however, the sender (sensor) and the receiver (controller) have different objectives… ▽ More We analyze in this paper finite horizon hierarchical signaling games between (information provider) senders and (decision maker) receivers in a dynamic environment. The underlying information evolves in time while sender and receiver interact repeatedly. Different from the classical communication (control) models, however, the sender (sensor) and the receiver (controller) have different objectives and there is a hierarchy between the players such that the sender leads the game by announcing his policies beforehand. He needs to anticipate the reaction of the receiver and the impact of the actions on the horizon while controlling the transparency of the disclosed information at each interaction. With quadratic cost functions and multivariate Gaussian processes, evolving according to first order auto-regressive models, we show that memoryless "linear" sender signaling rules are optimal (in the sense of game-theoretic hierarchical equilibrium) within the general class of measurable policies in the noncooperative communication context. In the noncooperative control context, we also analyze the hierarchical equilibrium for linear signaling rules and provide an algorithm to compute the optimal linear signaling rules numerically with global optimality guarantees. △ Less

Submitted 11 June, 2019; v1 submitted 29 September, 2016; originally announced September 2016.

Journal ref: Automatica, Volume 107, September 2019

Scalar Quadratic-Gaussian Soft Watermarking Games

Authors: Kivanc Mihcak, Emrah Akyol, Tamer Basar, Cedric Langbort

Abstract: We introduce the zero-sum game problem of soft watermarking: The hidden information (watermark) comes from a continuum and has a perceptual value; the receiver generates an estimate of the embedded watermark to minimize the expected estimation error (unlike the conventional watermarking schemes where both the hidden information and the receiver output are from a discrete finite set). Applications… ▽ More We introduce the zero-sum game problem of soft watermarking: The hidden information (watermark) comes from a continuum and has a perceptual value; the receiver generates an estimate of the embedded watermark to minimize the expected estimation error (unlike the conventional watermarking schemes where both the hidden information and the receiver output are from a discrete finite set). Applications include embedding a multimedia content into another. We consider in this paper the scalar Gaussian case and use expected mean-squared distortion. We formulate the resulting problem as a zero-sum game between the encoder & receiver pair and the attacker. We show that for the lin- ear encoder, the optimal attacker is Gaussian-affine, derive the optimal system parameters in that case, and discuss the corresponding system behavior. We also provide numerical results to gain further insight and understanding of the system behavior at optimality. △ Less

Submitted 12 July, 2016; originally announced July 2016.

Comments: submitted for publication

On the Structure of Equilibrium Strategies in Dynamic Gaussian Signaling Games

Authors: Muhammed Sayin, Emrah Akyol, Tamer Basar

Abstract: This paper analyzes a finite horizon dynamic signaling game motivated by the well-known strategic information transmission problems in economics. The mathematical model involves information transmission between two agents, a sender who observes two Gaussian processes, state and bias, and a receiver who takes an action based on the received message from the sender. The players incur quadratic insta… ▽ More This paper analyzes a finite horizon dynamic signaling game motivated by the well-known strategic information transmission problems in economics. The mathematical model involves information transmission between two agents, a sender who observes two Gaussian processes, state and bias, and a receiver who takes an action based on the received message from the sender. The players incur quadratic instantaneous costs as functions of the state, bias and action variables. Our particular focus is on the Stackelberg equilibrium, which corresponds to information disclosure and Bayesian persuasion problems in economics. Prior work solved the static game, and showed that the Stackelberg equilibrium is achieved by pure strategies that are linear functions of the state and the bias variables. The main focus of this work is on the dynamic (multi-stage) setting, where we show that the existence of a pure strategy Stackelberg equilibrium, within the set of linear strategies, depends on the problem parameters. Surprisingly, for most problem parameters, a pure linear strategy does not achieve the Stackelberg equilibrium which implies the existence of a trade-off between exploiting and revealing information, which was also encountered in several other asymmetric information games. △ Less

Submitted 11 July, 2016; originally announced July 2016.

Comments: will appear in IEEE Multi-Conference on Systems and Control 2016

On the Role of Side Information In Strategic Communication

Authors: Emrah Akyol, Cedric Langbort, Tamer Basar

Abstract: This paper analyzes the fundamental limits of strate- gic communication in network settings. Strategic communication differs from the conventional communication paradigms in in- formation theory since it involves different objectives for the encoder and the decoder, which are aware of this mismatch and act accordingly. This leads to a Stackelberg game where both agents commit to their map**s ex-… ▽ More This paper analyzes the fundamental limits of strate- gic communication in network settings. Strategic communication differs from the conventional communication paradigms in in- formation theory since it involves different objectives for the encoder and the decoder, which are aware of this mismatch and act accordingly. This leads to a Stackelberg game where both agents commit to their map**s ex-ante. Building on our prior work on the point-to-point setting, this paper studies the compression and communication problems with the receiver and/or transmitter side information setting. The equilibrium strategies and associated costs are characterized for the Gaussian variables with quadratic cost functions. Several questions on the benefit of side information in source and joint source-channel coding in such strategic settings are analyzed. Our analysis has uncovered an interesting result on optimality of uncoded map**s in strategic source-channel coding in networks. △ Less

Submitted 20 February, 2016; originally announced February 2016.

Comments: submitted to ISIT'16. arXiv admin note: text overlap with arXiv:1510.00764

Power-Distortion Metrics for Path Planning over Gaussian Sensor Networks

Authors: Emrah Akyol, Urbashi Mitra

Abstract: Path planning is an important component of au- tonomous mobile sensing systems. This paper studies upper and lower bounds of communication performance over Gaussian sen- sor networks, to drive power-distortion metrics for path planning problems. The Gaussian multiple-access channel is employed as a channel model and two source models are considered. In the first setting, the underlying source is e… ▽ More Path planning is an important component of au- tonomous mobile sensing systems. This paper studies upper and lower bounds of communication performance over Gaussian sen- sor networks, to drive power-distortion metrics for path planning problems. The Gaussian multiple-access channel is employed as a channel model and two source models are considered. In the first setting, the underlying source is estimated with minimum mean squared error, while in the second, reconstruction of a random spatial field is considered. For both problem settings, the upper and the lower bounds of sensor power-distortion curve are derived. For both settings, the upper bounds follow from the amplify-and-forward scheme and the lower bounds admit a unified derivation based on data processing inequality and tensorization property of the maximal correlation measure. Next, closed-form solutions of the optimal power allocation problems are obtained under a weighted sum-power constraint. The gap between the upper and the lower bounds is analyzed for both weighted sum and individual power constrained settings. Finally, these metrics are used to drive a path planning algorithm and the effects of power-distortion metrics, network parameters, and power optimization on the optimized path selection are analyzed. △ Less

Submitted 20 February, 2016; originally announced February 2016.

Comments: will appear in Trans. on Communications

Combinatorial Message Sharing and a New Achievable Region for Multiple Descriptions

Authors: Kumar Viswanatha, Emrah Akyol, Kenneth Rose

Abstract: This paper presents a new achievable rate-distortion region for the general L channel multiple descriptions problem. A well known general region for this problem is due to Venkataramani, Kramer and Goyal (VKG) [1]. Their encoding scheme is an extension of the El-Gamal-Cover (EC) and Zhang- Berger (ZB) coding schemes to the L channel case and includes a combinatorial number of refinement codebooks,… ▽ More This paper presents a new achievable rate-distortion region for the general L channel multiple descriptions problem. A well known general region for this problem is due to Venkataramani, Kramer and Goyal (VKG) [1]. Their encoding scheme is an extension of the El-Gamal-Cover (EC) and Zhang- Berger (ZB) coding schemes to the L channel case and includes a combinatorial number of refinement codebooks, one for each subset of the descriptions. As in ZB, the scheme also allows for a single common codeword to be shared by all descriptions. This paper proposes a novel encoding technique involving Combinatorial Message Sharing (CMS), where every subset of the descriptions may share a distinct common message. This introduces a combinatorial number of shared codebooks along with the refinement codebooks of [1]. We derive an achievable rate-distortion region for the proposed technique, and show that it subsumes the VKG region for general sources and distortion measures. We further show that CMS provides a strict improvement of the achievable region for any source and distortion measures for which some 2-description subset is such that ZB achieves points outside the EC region. We then show a more surprising result: CMS outperforms VKG for a general class of sources and distortion measures, including scenarios where the ZB and EC regions coincide for all 2-description subsets. In particular, we show that CMS strictly improves on VKG, for the L-channel quadratic Gaussian MD problem, for all L greater than or equal to 3, despite the fact that the EC region is complete for the corresponding 2-descriptions problem. Using the encoding principles derived, we show that the CMS scheme achieves the complete rate-distortion region for several asymmetric cross-sections of the L-channel quadratic Gaussian MD problem. △ Less

Submitted 6 December, 2015; originally announced December 2015.

Deterministic Annealing Based Optimization for Zero-Delay Source-Channel Coding in Networks

Authors: Mustafa Said Mehmetoglu, Emrah Akyol, Kenneth Rose

Abstract: This paper studies the problem of global optimization of zero-delay source-channel codes that map between the source space and the channel space, under a given transmission power constraint and for the mean square error distortion. Particularly, we focus on two well known network settings: the Wyner-Ziv setting where only a decoder has access to side information and the distributed setting where i… ▽ More This paper studies the problem of global optimization of zero-delay source-channel codes that map between the source space and the channel space, under a given transmission power constraint and for the mean square error distortion. Particularly, we focus on two well known network settings: the Wyner-Ziv setting where only a decoder has access to side information and the distributed setting where independent encoders transmit over independent channels to a central decoder. Prior work derived the necessary conditions for optimality of the encoder and decoder map**s, along with a greedy optimization algorithm that imposes these conditions iteratively, in conjunction with the heuristic noisy channel relaxation method to mitigate poor local minima. While noisy channel relaxation is arguably effective in simple settings, it fails to provide accurate global optimization in more complicated settings considered in this paper. We propose a powerful non-convex optimization method based on the concept of deterministic annealing -- which is derived from information theoretic principles and was successfully employed in several problems including vector quantization, classification and regression. We present comparative numerical results that show strict superiority of the proposed method over greedy optimization methods as well as prior approaches in literature. △ Less

Submitted 6 November, 2015; originally announced November 2015.

Comments: Accepted for publication at IEEE Transactions on Communications

Privacy Constrained Information Processing

Authors: Emrah Akyol, Cedric Langbort, Tamer Basar

Abstract: This paper studies communication scenarios where the transmitter and the receiver have different objectives due to privacy concerns, in the context of a variation of the strategic information transfer (SIT) model of Sobel and Crawford. We first formulate the problem as the minimization of a common distortion by the transmitter and the receiver subject to a privacy constrained transmitter. We show… ▽ More This paper studies communication scenarios where the transmitter and the receiver have different objectives due to privacy concerns, in the context of a variation of the strategic information transfer (SIT) model of Sobel and Crawford. We first formulate the problem as the minimization of a common distortion by the transmitter and the receiver subject to a privacy constrained transmitter. We show the equivalence of this formulation to a Stackelberg equilibrium of the SIT problem. Assuming an entropy based privacy measure, a quadratic distortion measure and jointly Gaussian variables, we characterize the Stackelberg equilibrium. Next, we consider asymptotically optimal compression at the transmitter which inherently provides some level of privacy, and study equilibrium conditions. We finally analyze the impact of the presence of an average power constrained Gaussian communication channel between the transmitter and the receiver on the equilibrium conditions. △ Less

Submitted 12 October, 2015; originally announced October 2015.

Comments: will appear in CDC'15

Information-Theoretic Approach to Strategic Communication as a Hierarchical Game

Authors: Emrah Akyol, Cedric Langbort, Tamer Basar

Abstract: This paper analyzes the information disclosure problems originated in economics through the lens of information theory. Such problems are radically different from the conventional communication paradigms in information theory since they involve different objectives for the encoder and the decoder, which are aware of this mismatch and act accordingly. This leads, in our setting, to a hierarchical c… ▽ More This paper analyzes the information disclosure problems originated in economics through the lens of information theory. Such problems are radically different from the conventional communication paradigms in information theory since they involve different objectives for the encoder and the decoder, which are aware of this mismatch and act accordingly. This leads, in our setting, to a hierarchical communication game, where the transmitter announces an encoding strategy with full commitment, and its distortion measure depends on a private information sequence whose realization is available at the transmitter. The receiver decides on its decoding strategy that minimizes its own distortion based on the announced encoding map and the statistics. Three problem settings are considered, focusing on the quadratic distortion measures, and jointly Gaussian source and private information: compression, communication, and the simple equilibrium conditions without any compression or communication. The equilibrium strategies and associated costs are characterized. The analysis is then extended to the receiver side information setting and the major changes in the structure of optimal strategies are identified. Finally, an extension of the results to the broader context of decentralized stochastic control is presented. △ Less

Submitted 11 July, 2016; v1 submitted 2 October, 2015; originally announced October 2015.

Comments: in press, the Proceedings of the IEEE, Special Issue on Principles and Applications of Science of Information

Analog Multiple Descriptions: A Zero-Delay Source-Channel Coding Approach

Authors: Mustafa Said Mehmetoglu, Emrah Akyol, Kenneth Rose

Abstract: This paper extends the well-known source coding problem of multiple descriptions, in its general and basic setting, to analog source-channel coding scenarios. Encoding-decoding functions that optimally map between the (possibly continuous valued) source and the channel spaces are numerically derived. The main technical tool is a non-convex optimization method, namely, deterministic annealing, whic… ▽ More This paper extends the well-known source coding problem of multiple descriptions, in its general and basic setting, to analog source-channel coding scenarios. Encoding-decoding functions that optimally map between the (possibly continuous valued) source and the channel spaces are numerically derived. The main technical tool is a non-convex optimization method, namely, deterministic annealing, which has recently been successfully used in other map** optimization problems. The obtained functions exhibit several interesting structural properties, map multiple source intervals to the same interval in the channel space, and consistently outperform the known competing map** techniques. △ Less

Submitted 1 October, 2015; originally announced October 2015.

Comments: Submitted to ICASSP 2016

Optimal Sensor Scheduling and Remote Estimation over an Additive Noise Channel

Authors: Xiaobin Gao, Emrah Akyol, Tamer Basar

Abstract: We consider a sensor scheduling and remote estimation problem with one sensor and one estimator. At each time step, the sensor makes an observation on the state of a source, and then decides whether to transmit its observation to the estimator or not. The sensor is charged a cost for each transmission. The remote estimator generates a real-time estimate on the state of the source based on the mess… ▽ More We consider a sensor scheduling and remote estimation problem with one sensor and one estimator. At each time step, the sensor makes an observation on the state of a source, and then decides whether to transmit its observation to the estimator or not. The sensor is charged a cost for each transmission. The remote estimator generates a real-time estimate on the state of the source based on the messages received from the sensor. The estimator is charged for estimation error. As compared with previous works from the literature, we further assume that there is an additive communication channel noise. As a consequence, the sensor needs to encode the message before transmitting it to the estimator. For some specific distributions of the underlying random variables, we obtain the optimal solution to the problem of minimizing the expected value of the sum of communication cost and estimation cost over the time horizon. △ Less

Submitted 30 September, 2015; originally announced October 2015.

Comments: In Proceedings of American Control Conference (ACC), pages 2723-2728, 2015

On Remote Estimation with Multiple Communication Channels

Authors: Xiaobin Gao, Emrah Akyol, Tamer Basar

Abstract: This paper considers a sequential estimation and sensor scheduling problem in the presence of multiple communication channels. As opposed to the classical remote estimation problem that involves one perfect (noiseless) channel and one extremely noisy channel (which corresponds to not transmitting the observed state), a more realistic additive noise channel with fixed power constraint along with a… ▽ More This paper considers a sequential estimation and sensor scheduling problem in the presence of multiple communication channels. As opposed to the classical remote estimation problem that involves one perfect (noiseless) channel and one extremely noisy channel (which corresponds to not transmitting the observed state), a more realistic additive noise channel with fixed power constraint along with a more costly perfect channel is considered. It is shown, via a counter-example, that the common folklore of applying symmetric threshold policy, which is well known to be optimal (for unimodal state densities) in the classical two-channel remote estimation problem, can be suboptimal for the setting considered. Next, in order to make the problem tractable, a side channel which signals the sign of the underlying state is considered. It is shown that, under some technical assumptions, threshold-in-threshold communication scheduling is optimal for this setting. The impact of the presence of a noisy channel is analyzed numerically based on dynamic programming. This numerical analysis uncovers some rather surprising results inheriting known properties from the noisy and noiseless settings. △ Less

Submitted 30 September, 2015; originally announced October 2015.

Comments: Submitted to 2016 American Control Conference (ACC 2016)

Optimal Estimation with Limited Measurements and Noisy Communication

Authors: Xiaobin Gao, Emrah Akyol, Tamer Basar

Abstract: This paper considers a sequential estimation and sensor scheduling problem with one sensor and one estimator. The sensor makes sequential observations about the state of an underlying memoryless stochastic process, and makes a decision as to whether or not to send this measurement to the estimator. The sensor and the estimator have the common objective of minimizing expected distortion in the esti… ▽ More This paper considers a sequential estimation and sensor scheduling problem with one sensor and one estimator. The sensor makes sequential observations about the state of an underlying memoryless stochastic process, and makes a decision as to whether or not to send this measurement to the estimator. The sensor and the estimator have the common objective of minimizing expected distortion in the estimation of the state of the process, over a finite time horizon, with the constraint that the sensor can transmit its observation only a limited number of times. As opposed to the prior work where communication between the sensor and the estimator was assumed to be perfect (noiseless), in this work an additive noise channel with fixed power constraint is considered; hence, the sensor has to encode its message before transmission. For some specific source and channel noise densities, we obtain the optimal encoding and estimation policies in conjunction with the optimal transmission schedule. The impact of the presence of a noisy channel is analyzed numerically based on dynamic programming. This analysis yields some rather surprising results such as a phase-transition phenomenon in the number of used transmission opportunities, which was not encountered in the noiseless communication setting. △ Less

Submitted 28 September, 2015; originally announced September 2015.

Comments: X. Gao, E. Akyol, and T. Basar. Optimal estimation with limited measurements and noisy communication. In 54th IEEE Conference on Decision and Control (CDC15), 2015, to appear

The Lossy Common Information of Correlated Sources

Authors: Kumar Viswanatha, Emrah Akyol, Kenneth Rose

Abstract: The two most prevalent notions of common information (CI) are due to Wyner and Gacs-Korner and both the notions can be stated as two different characteristic points in the lossless Gray-Wyner region. Although the information theoretic characterizations for these two CI quantities can be easily evaluated for random variables with infinite entropy (eg., continuous random variables), their operationa… ▽ More The two most prevalent notions of common information (CI) are due to Wyner and Gacs-Korner and both the notions can be stated as two different characteristic points in the lossless Gray-Wyner region. Although the information theoretic characterizations for these two CI quantities can be easily evaluated for random variables with infinite entropy (eg., continuous random variables), their operational significance is applicable only to the lossless framework. The primary objective of this paper is to generalize these two CI notions to the lossy Gray-Wyner network, which hence extends the theoretical foundation to general sources and distortion measures. We begin by deriving a single letter characterization for the lossy generalization of Wyner's CI, defined as the minimum rate on the shared branch of the Gray-Wyner network, maintaining minimum sum transmit rate when the two decoders reconstruct the sources subject to individual distortion constraints. To demonstrate its use, we compute the CI of bivariate Gaussian random variables for the entire regime of distortions. We then similarly generalize Gacs and Korner's definition to the lossy framework. The latter half of the paper focuses on studying the tradeoff between the total transmit rate and receive rate in the Gray-Wyner network. We show that this tradeoff yields a contour of points on the surface of the Gray-Wyner region, which passes through both the Wyner and Gacs-Korner operating points, and thereby provides a unified framework to understand the different notions of CI. We further show that this tradeoff generalizes the two notions of CI to the excess sum transmit rate and receive rate regimes, respectively. △ Less

Submitted 31 March, 2014; originally announced March 2014.

A Deterministic Annealing Optimization Approach for Witsenhausen's and Related Decentralized Control Settings

Authors: Mustafa Mehmetoglu, Emrah Akyol, Kenneth Rose

Abstract: This paper studies the problem of map** optimization in decentralized control problems. A global optimization algorithm is proposed based on the ideas of ``deterministic annealing" - a powerful non-convex optimization framework derived from information theoretic principles with analogies to statistical physics. The key idea is to randomize the map**s and control the Shannon entropy of the syst… ▽ More This paper studies the problem of map** optimization in decentralized control problems. A global optimization algorithm is proposed based on the ideas of ``deterministic annealing" - a powerful non-convex optimization framework derived from information theoretic principles with analogies to statistical physics. The key idea is to randomize the map**s and control the Shannon entropy of the system during optimization. The entropy constraint is gradually relaxed in a deterministic annealing process while tracking the minimum, to obtain the ultimate deterministic map**s. Deterministic annealing has been successfully employed in several problems including clustering, vector quantization, regression, as well as the Witsenhausen's counterexample in our recent work[1]. We extend our method to a more involved setting, a variation of Witsenhausen's counterexample, where there is a side channel between the two controllers. The problem can be viewed as a two stage cancellation problem. We demonstrate that there exist complex strategies that can exploit the side channel efficiently, obtaining significant gains over the best affine and known non-linear strategies. △ Less

Submitted 20 March, 2014; originally announced March 2014.

Comments: submitted to CDC'14

A Deterministic Annealing Approach to Witsenhausen's Counterexample

Authors: Mustafa Mehmetoglu, Emrah Akyol, Kenneth Rose

Abstract: This paper proposes a numerical method, based on information theoretic ideas, to a class of distributed control problems. As a particular test case, the well-known and numerically "over-mined" problem of decentralized control and implicit communication, commonly referred to as Witsenhausen's counterexample, is considered. The method provides a small improvement over the best numerical result so fa… ▽ More This paper proposes a numerical method, based on information theoretic ideas, to a class of distributed control problems. As a particular test case, the well-known and numerically "over-mined" problem of decentralized control and implicit communication, commonly referred to as Witsenhausen's counterexample, is considered. The method provides a small improvement over the best numerical result so far for this benchmark problem. The key idea is to randomize the zero-delay map**s. which become "soft", probabilistic map**s to be optimized in a deterministic annealing process, by incorporating a Shannon entropy constraint in the problem formulation. The entropy of the map** is controlled and gradually lowered to zero to obtain deterministic map**s, while avoiding poor local minima. Proposed method obtains new map**s that shed light on the structure of the optimal solution, as well as achieving a small improvement in total cost over the state of the art in numerical approaches to this problem. △ Less

Submitted 3 February, 2014; originally announced February 2014.

Comments: submitted to ISIT'14

Optimization of zero-delay map**s for distributed coding by deterministic annealing

Authors: Mustafa S. Mehmetoglu, Emrah Akyol, Kenneth Rose

Abstract: This paper studies the optimization of zero-delay analog map**s in a network setting that involves distributed coding. The cost surface is known to be non-convex, and known greedy methods tend to get trapped in poor locally optimal solutions that depend heavily on initialization. We derive an optimization algorithm based on the principles of "deterministic annealing", a powerful global optimizat… ▽ More This paper studies the optimization of zero-delay analog map**s in a network setting that involves distributed coding. The cost surface is known to be non-convex, and known greedy methods tend to get trapped in poor locally optimal solutions that depend heavily on initialization. We derive an optimization algorithm based on the principles of "deterministic annealing", a powerful global optimization framework that has been successfully employed in several disciplines, including, in our recent work, to a simple zero-delay analog communications problem. We demonstrate strict superiority over the descent based methods, as well as present example map**s whose properties lend insights on the workings of the solution and relations with digital distributed coding. △ Less

Submitted 2 December, 2013; originally announced December 2013.

Gaussian Sensor Networks with Adversarial Nodes

Authors: Emrah Akyol, Kenneth Rose, Tamer Basar

Abstract: This paper studies a particular sensor network model which involves one single Gaussian source observed by many sensors, subject to additive independent Gaussian observation noise. Sensors communicate with the receiver over an additive Gaussian multiple access channel. The aim of the receiver is to reconstruct the underlying source with minimum mean squared error. The scenario of interest here is… ▽ More This paper studies a particular sensor network model which involves one single Gaussian source observed by many sensors, subject to additive independent Gaussian observation noise. Sensors communicate with the receiver over an additive Gaussian multiple access channel. The aim of the receiver is to reconstruct the underlying source with minimum mean squared error. The scenario of interest here is one where some of the sensors act as adversary (jammer): they strive to maximize distortion. We show that the ability of transmitter sensors to secretly agree on a random event, that is "coordination", plays a key role in the analysis. Depending on the coordination capability of sensors and the receiver, we consider two problem settings. The first setting involves transmitters with coordination capabilities in the sense that all transmitters can use identical realization of randomized encoding for each transmission. In this case, the optimal strategy for the adversary sensors also requires coordination, where they all generate the same realization of independent and identically distributed Gaussian noise. In the second setting, the transmitter sensors are restricted to use fixed, deterministic encoders and this setting, which corresponds to a Stackelberg game, does not admit a saddle-point solution. We show that the the optimal strategy for all sensors is uncoded communications where encoding functions of adversaries and transmitters are in opposite directions. For both settings, digital compression and communication is strictly suboptimal. △ Less

Submitted 16 May, 2013; originally announced May 2013.

Comments: 5 pages, will be presented at ISIT 2013, Istanbul, Turkey

On the Role of Common Codewords in Quadratic Gaussian Multiple Descriptions Coding

Authors: Kumar Viswanatha, Emrah Akyol, Kenneth Rose

Abstract: This paper focuses on the problem of $L-$channel quadratic Gaussian multiple description (MD) coding. We recently introduced a new encoding scheme in [1] for general $L-$channel MD problem, based on a technique called `Combinatorial Message Sharing' (CMS), where every subset of the descriptions shares a distinct common message. The new achievable region subsumes the most well known region for the… ▽ More This paper focuses on the problem of $L-$channel quadratic Gaussian multiple description (MD) coding. We recently introduced a new encoding scheme in [1] for general $L-$channel MD problem, based on a technique called `Combinatorial Message Sharing' (CMS), where every subset of the descriptions shares a distinct common message. The new achievable region subsumes the most well known region for the general problem, due to Venkataramani, Kramer and Goyal (VKG) [2]. Moreover, we showed in [3] that the new scheme provides a strict improvement of the achievable region for any source and distortion measures for which some 2-description subset is such that the Zhang and Berger (ZB) scheme achieves points outside the El-Gamal and Cover (EC) region. In this paper, we show a more surprising result: CMS outperforms VKG for a general class of sources and distortion measures, which includes scenarios where for all 2-description subsets, the ZB and EC regions coincide. In particular, we show that CMS strictly extends VKG region, for the $L$-channel quadratic Gaussian MD problem for all $L\geq3$, despite the fact that the EC region is complete for the corresponding 2-descriptions problem. Using the encoding principles derived, we show that the CMS scheme achieves the complete rate-distortion region for several asymmetric cross-sections of the $L-$channel quadratic Gaussian MD problem, which have not been considered earlier. △ Less

Submitted 15 May, 2013; originally announced May 2013.

A Deterministic Annealing Approach to Optimization of Zero-delay Source-Channel Codes

Authors: Mustafa S. Mehmetoglu, Emrah Akyol, Kenneth Rose

Abstract: This paper studies optimization of zero-delay source-channel codes, and specifically the problem of obtaining globally optimal transformations that map between the source space and the channel space, under a given transmission power constraint and for the mean square error distortion. Particularly, we focus on the setting where the decoder has access to side information, whose cost surface is know… ▽ More This paper studies optimization of zero-delay source-channel codes, and specifically the problem of obtaining globally optimal transformations that map between the source space and the channel space, under a given transmission power constraint and for the mean square error distortion. Particularly, we focus on the setting where the decoder has access to side information, whose cost surface is known to be riddled with local minima. Prior work derived the necessary conditions for optimality of the encoder and decoder map**s, along with a greedy optimization algorithm that imposes these conditions iteratively, in conjunction with the heuristic "noisy channel relaxation" method to mitigate poor local minima. While noisy channel relaxation is arguably effective in simple settings, it fails to provide accurate global optimization results in more complicated settings including the decoder with side information as considered in this paper. We propose a global optimization algorithm based on the ideas of "deterministic annealing"- a non-convex optimization method, derived from information theoretic principles with analogies to statistical physics, and successfully employed in several problems including clustering, vector quantization and regression. We present comparative numerical results that show strict superiority of the proposed algorithm over greedy optimization methods as well as over the noisy channel relaxation. △ Less

Submitted 25 April, 2013; originally announced April 2013.

Comments: Submitted to ITW 2013, in review

On Optimal Jamming Over an Additive Noise Channel

Authors: Emrah Akyol, Kenneth Rose, Tamer Basar

Abstract: This paper considers the problem of optimal zero-delay jamming over an additive noise channel. Early work had already solved this problem for a Gaussian source and channel. Building on a sequence of recent results on conditions for linearity of optimal estimation, and of optimal map**s in source-channel coding, we derive the saddle-point solution to the jamming problem for general sources and ch… ▽ More This paper considers the problem of optimal zero-delay jamming over an additive noise channel. Early work had already solved this problem for a Gaussian source and channel. Building on a sequence of recent results on conditions for linearity of optimal estimation, and of optimal map**s in source-channel coding, we derive the saddle-point solution to the jamming problem for general sources and channels, without recourse to Gaussian assumptions. We show that linearity conditions play a pivotal role in jamming, in the sense that the optimal jamming strategy is to effectively force both transmitter and receiver to default to linear map**s, i.e., the jammer ensures, whenever possible, that the transmitter and receiver cannot benefit from non-linear strategies. This result is shown to subsume the known result for Gaussian source and channel. We analyze conditions and general settings where such unbeatable strategy can indeed be achieved by the jammer. Moreover, we provide the procedure to approximate optimal jamming in the remaining (source-channel) cases where the jammer cannot impose linearity on the transmitter and the receiver. △ Less

Submitted 12 March, 2013; originally announced March 2013.

Comments: Submitted to 2013 Conference on Decision and Control

On Zero Delay Source-Channel Coding

Authors: Emrah Akyol, Kumar Viswanatha, Kenneth Rose, Tor Ramstad

Abstract: In this paper, we study the zero-delay source-channel coding problem, and specifically the problem of obtaining the vector transformations that optimally map between the m-dimensional source space and the k-dimensional channel space, under a given transmission power constraint and for the mean square error distortion. We first study the functional properties of this problem and show that the objec… ▽ More In this paper, we study the zero-delay source-channel coding problem, and specifically the problem of obtaining the vector transformations that optimally map between the m-dimensional source space and the k-dimensional channel space, under a given transmission power constraint and for the mean square error distortion. We first study the functional properties of this problem and show that the objective is concave in the source and noise densities and convex in the density of the input to the channel. We then derive the necessary conditions for optimality of the encoder and decoder map**s. A well known result in information theory pertains to the linearity of optimal encoding and decoding map**s in the scalar Gaussian source and channel setting, at all channel signal-to-noise ratios (CSNRs). In this paper, we study this result more generally, beyond the Gaussian source and channel, and derive the necessary and sufficient condition for linearity of optimal map**s, given a noise (or source) distribution, and a specified power constraint. We also prove that the Gaussian source-channel pair is unique in the sense that it is the only source-channel pair for which the optimal map**s are linear at more than one CSNR values. Moreover, we show the asymptotic linearity of optimal map**s for low CSNR if the channel is Gaussian regardless of the source and, at the other extreme, for high CSNR if the source is Gaussian, regardless of the channel. Our numerical results show strict improvement over prior methods. The numerical approach is extended to the scenario of source-channel coding with decoder side information. The resulting encoding map**s are shown to be continuous relatives of, and in fact subsume as special case, the Wyner-Ziv map**s encountered in digital distributed source coding systems. △ Less

Submitted 14 February, 2013; originally announced February 2013.

Comments: Submitted to IEEE Transactions on Information Theory, 18 pages, 10 figures

On Large Scale Distributed Compression and Dispersive Information Routing for Networks

Authors: Kumar Viswanatha, Sharadh Ramaswamy, Ankur Saxena, Emrah Akyol, Kenneth Rose

Abstract: This paper considers the problem of distributed source coding for a large network. A major obstacle that poses an existential threat to practical deployment of conventional approaches to distributed coding is the exponential growth of the decoder complexity with the number of sources and the encoding rates. This growth in complexity renders many traditional approaches impractical even for moderate… ▽ More This paper considers the problem of distributed source coding for a large network. A major obstacle that poses an existential threat to practical deployment of conventional approaches to distributed coding is the exponential growth of the decoder complexity with the number of sources and the encoding rates. This growth in complexity renders many traditional approaches impractical even for moderately sized networks. In this paper, we propose a new decoding paradigm for large scale distributed compression wherein the decoder complexity is explicitly controlled during the design. Central to our approach is a module called the "bit-subset selector" whose role is to judiciously extract an appropriate subset of the received bits for decoding per individual source. We propose a practical design strategy, based on deterministic annealing (DA) for the joint design of the system components, that enables direct optimization of the decoder complexity-distortion trade-off, and thereby the desired scalability. We also point out the direct connections between the problem of large scale distributed compression and a related problem in sensor networks, namely, dispersive information routing of correlated sources. This allows us to extend the design principles proposed in the context of large scale distributed compression to design efficient routers for minimum cost communication of correlated sources across a network. Experiments on both real and synthetic data-sets provide evidence for substantial gains over conventional approaches. △ Less

Submitted 5 January, 2013; originally announced January 2013.

Minimum Communication Cost for Joint Distributed Source Coding and Dispersive Information Routing

Authors: Kumar Viswanatha, Emrah Akyol, Kenneth Rose

Abstract: This paper considers the problem of minimum cost communication of correlated sources over a network with multiple sinks, which consists of distributed source coding followed by routing. We introduce a new routing paradigm called dispersive information routing, wherein the intermediate nodes are allowed to `split' a packet and forward subsets of the received bits on each of the forward paths. This… ▽ More This paper considers the problem of minimum cost communication of correlated sources over a network with multiple sinks, which consists of distributed source coding followed by routing. We introduce a new routing paradigm called dispersive information routing, wherein the intermediate nodes are allowed to `split' a packet and forward subsets of the received bits on each of the forward paths. This paradigm opens up a rich class of research problems which focus on the interplay between encoding and routing in a network. Unlike conventional routing methods such as in [1], dispersive information routing ensures that each sink receives just the information needed to reconstruct the sources it is required to reproduce. We demonstrate using simple examples that our approach offers better asymptotic performance than conventional routing techniques. This paradigm leads to a new information theoretic setup, which has not been studied earlier. We propose a new coding scheme, using principles from multiple descriptions encoding [2] and Han and Kobayashi decoding [3]. We show that this coding scheme achieves the complete rate region for certain special cases of the general setup and thereby achieves the minimum communication cost under this routing paradigm. △ Less

Submitted 19 September, 2012; originally announced September 2012.

Towards Optimality in Transform Coding

Authors: Emrah Akyol, Kenneth Rose

Abstract: It is well-known for transform coding of multivariate Gaussian sources, that the Karhunen-Loève transform (KLT) minimizes the mean square error distortion. However, finding the optimal transform for general non-Gaussian sources has been an open problem for decades, despite several important advances that provide some partial answers regarding KLT optimality. In this paper, we present a necessary a… ▽ More It is well-known for transform coding of multivariate Gaussian sources, that the Karhunen-Loève transform (KLT) minimizes the mean square error distortion. However, finding the optimal transform for general non-Gaussian sources has been an open problem for decades, despite several important advances that provide some partial answers regarding KLT optimality. In this paper, we present a necessary and sufficient condition for optimality of a transform when high resolution, variable rate quantizers are employed. We hence present not only a complete characterization of when KLT is optimal, but also a determining condition for optimality of a general (non-KLT) transform. This necessary and sufficient condition is shown to have direct connections to the well studied source separation problem. This observation can impact source separation itself, as illustrated with a new optimality result. We combine the transform optimality condition with algorithmic tools from source separation, to derive a practical numerical method to search for the optimal transform in source coding. Then, we focus on multiterminal settings, for which {\it conditional} KLT was shown to possess certain optimality properties for Gaussian sources. We derive the optimal orthogonal transform for the setting where side information is only available to the decoder, along with new specialized results specific to the conditions for optimality of conditional KLT. Finally, we consider distributed source coding where two correlated sources are to be transform coded separately but decoded jointly. We derive the necessary and sufficient condition of optimality of the orthogonal transforms. We specialize to find the optimal orthogonal transforms, in this setting, for specific source densities, including jointly Gaussian sources. △ Less

Submitted 18 June, 2012; v1 submitted 14 June, 2012; originally announced June 2012.

Comments: discovered a related prior work on the subject

On Constrained Randomized Quantization

Authors: Emrah Akyol, Kenneth Rose

Abstract: Randomized (dithered) quantization is a method capable of achieving white reconstruction error independent of the source. Dithered quantizers have traditionally been considered within their natural setting of uniform quantization. In this paper we extend conventional dithered quantization to nonuniform quantization, via a subterfage: dithering is performed in the companded domain. Closed form nece… ▽ More Randomized (dithered) quantization is a method capable of achieving white reconstruction error independent of the source. Dithered quantizers have traditionally been considered within their natural setting of uniform quantization. In this paper we extend conventional dithered quantization to nonuniform quantization, via a subterfage: dithering is performed in the companded domain. Closed form necessary conditions for optimality of the compressor and expander map**s are derived for both fixed and variable rate randomized quantization. Numerically, map**s are optimized by iteratively imposing these necessary conditions. The framework is extended to include an explicit constraint that deterministic or randomized quantizers yield reconstruction error that is uncorrelated with the source. Surprising theoretical results show direct and simple connection between the optimal constrained quantizers and their unconstrained counterparts. Numerical results for the Gaussian source provide strong evidence that the proposed constrained randomized quantizer outperforms the conventional dithered quantizer, as well as the constrained deterministic quantizer. Moreover, the proposed constrained quantizer renders the reconstruction error nearly white. In the second part of the paper, we investigate whether uncorrelated reconstruction error requires random coding to achieve asymptotic optimality. We show that for a Gaussian source, the optimal vector quantizer of asymptotically high dimension whose quantization error is uncorrelated with the source, is indeed random. Thus, random encoding in this setting of rate-distortion theory, is not merely a tool to characterize performance bounds, but a required property of quantizers that approach such bounds. △ Less

Submitted 13 June, 2012; originally announced June 2012.

Comments: submitted to IEEE Transactions on Signal Processing, 2012

Subset Typicality Lemmas and Improved Achievable Regions in Multiterminal Source Coding

Authors: Kumar Viswanatha, Emrah Akyol, Kenneth Rose

Abstract: Consider the following information theoretic setup wherein independent codebooks of N correlated random variables are generated according to their respective marginals. The problem of determining the conditions on the rates of codebooks to ensure the existence of at least one codeword tuple which is jointly typical with respect to a given joint density (called the multivariate covering lemma) has… ▽ More Consider the following information theoretic setup wherein independent codebooks of N correlated random variables are generated according to their respective marginals. The problem of determining the conditions on the rates of codebooks to ensure the existence of at least one codeword tuple which is jointly typical with respect to a given joint density (called the multivariate covering lemma) has been studied fairly well and the associated rate regions have found applications in several source coding scenarios. However, several multiterminal source coding applications, such as the general multi-user Gray-Wyner network, require joint typicality only within subsets of codewords transmitted. Motivated by such applications, we ask ourselves the conditions on the rates to ensure the existence of at least one codeword tuple which is jointly typical within subsets according to given per subset joint densities. This report focuses primarily on deriving a new achievable rate region for this problem which strictly improves upon the direct extension of the multivariate covering lemma, which has quite popularly been used in several earlier work. Towards proving this result, we derive two important results called `subset typicality lemmas' which can potentially have broader applicability in more general scenarios beyond what is considered in this report. We finally apply the results therein to derive a new achievable region for the general multi-user Gray-Wyner network. △ Less

Submitted 5 May, 2012; originally announced May 2012.

A Strictly Improved Achievable Region for Multiple Descriptions Using Combinatorial Message Sharing

Authors: Kumar Viswanatha, Emrah Akyol, Kenneth Rose

Abstract: We recently proposed a new coding scheme for the L-channel multiple descriptions (MD) problem for general sources and distortion measures involving `Combinatorial Message Sharing' (CMS) [7] leading to a new achievable rate-distortion region. Our objective in this paper is to establish that this coding scheme strictly subsumes the most popular region for this problem due to Venkataramani, Kramer an… ▽ More We recently proposed a new coding scheme for the L-channel multiple descriptions (MD) problem for general sources and distortion measures involving `Combinatorial Message Sharing' (CMS) [7] leading to a new achievable rate-distortion region. Our objective in this paper is to establish that this coding scheme strictly subsumes the most popular region for this problem due to Venkataramani, Kramer and Goyal (VKG) [3]. In particular, we show that for a binary symmetric source under Hamming distortion measure, the CMS scheme provides a strictly larger region for all L>2. The principle of the CMS coding scheme is to include a common message in every subset of the descriptions, unlike the VKG scheme which sends a single common message in all the descriptions. In essence, we show that allowing for a common codeword in every subset of descriptions provides better freedom in coordinating the messages which can be exploited constructively to achieve points outside the VKG region. △ Less

Submitted 30 May, 2011; originally announced May 2011.

On Conditions for Linearity of Optimal Estimation

Authors: Emrah Akyol, Kumar Viswanatha, Kenneth Rose

Abstract: When is optimal estimation linear? It is well known that, when a Gaussian source is contaminated with Gaussian noise, a linear estimator minimizes the mean square estimation error. This paper analyzes, more generally, the conditions for linearity of optimal estimators. Given a noise (or source) distribution, and a specified signal to noise ratio (SNR), we derive conditions for existence and unique… ▽ More When is optimal estimation linear? It is well known that, when a Gaussian source is contaminated with Gaussian noise, a linear estimator minimizes the mean square estimation error. This paper analyzes, more generally, the conditions for linearity of optimal estimators. Given a noise (or source) distribution, and a specified signal to noise ratio (SNR), we derive conditions for existence and uniqueness of a source (or noise) distribution for which the $L_p$ optimal estimator is linear. We then show that, if the noise and source variances are equal, then the matching source must be distributed identically to the noise. Moreover, we prove that the Gaussian source-channel pair is unique in the sense that it is the only source-channel pair for which the mean square error (MSE) optimal estimator is linear at more than one SNR values. Further, we show the asymptotic linearity of MSE optimal estimators for low SNR if the channel is Gaussian regardless of the source and, vice versa, for high SNR if the source is Gaussian regardless of the channel. The extension to the vector case is also considered where besides the conditions inherited from the scalar case, additional constraints must be satisfied to ensure linearity of the optimal estimator. △ Less

Submitted 27 February, 2012; v1 submitted 6 April, 2011; originally announced April 2011.

Comments: To appear in IEEE Transactions on Information Theory, accepted on Feb 5, 2012