Mathematics of Sparsity and Entropy: Axioms, Core Functions and Sparse Recovery
Authors:
Giancarlo Pastor,
Inmaculada Mora-Jiménez,
Riku Jäntti,
Antonio J. Caamaño
Abstract:
Sparsity and entropy are pillar notions of modern theories in signal processing and information theory. However, there is no clear consensus among scientists on the characterization of these notions. Previous efforts have contributed to understand individually sparsity or entropy from specific research interests. This paper proposes a mathematical formalism, a joint axiomatic characterization, whi…
▽ More
Sparsity and entropy are pillar notions of modern theories in signal processing and information theory. However, there is no clear consensus among scientists on the characterization of these notions. Previous efforts have contributed to understand individually sparsity or entropy from specific research interests. This paper proposes a mathematical formalism, a joint axiomatic characterization, which contributes to comprehend (the beauty of) sparsity and entropy. The paper gathers and introduces inherent and first principles criteria as axioms and attributes that jointly characterize sparsity and entropy. The proposed set of axioms is constructive and allows to derive simple or \emph{core functions} and further generalizations. Core sparsity generalizes the Hoyer measure, Gini index and $pq$-means. Core entropy generalizes the Rényi entropy and Tsallis entropy, both of which generalize Shannon entropy. Finally, core functions are successfully applied to compressed sensing and to minimum entropy given sample moments. More importantly, the (simplest) core sparsity adds theoretical support to the $\ell_1$-minimization approach in compressed sensing.
△ Less
Submitted 16 December, 2015; v1 submitted 21 January, 2015;
originally announced January 2015.
A Theory of Routing for Large-Scale Wireless Ad-Hoc Networks
Authors:
Antonio J. Caamaño,
Juan J. Vinagre,
Mark Wilby,
Javier Ramos
Abstract:
In this work we develop a new theory to analyse the process of routing in large-scale ad-hoc wireless networks. We use a path integral formulation to examine the properties of the paths generated by different routing strategies in these kinds of networks. Using this theoretical framework, we calculate the statistical distribution of the distances between any source to any destination in the netw…
▽ More
In this work we develop a new theory to analyse the process of routing in large-scale ad-hoc wireless networks. We use a path integral formulation to examine the properties of the paths generated by different routing strategies in these kinds of networks. Using this theoretical framework, we calculate the statistical distribution of the distances between any source to any destination in the network, hence we are able to deduce a length parameter that is unique for each routing strategy. This parameter, defined as the {\it effective radius}, effectively encodes the routing information required by a node. Analysing the aforementioned statistical distribution for different routing strategies, we obtain a threefold result for practical Large-Scale Wireless Ad-Hoc Networks: 1) We obtain the distribution of the lengths of all the paths in a network for any given routing strategy, 2) We are able to identify "good" routing strategies depending on the evolution of its effective radius as the number of nodes, $N$, increases to infinity, 3) For any routing strategy with finite effective radius, we demonstrate that, in a large-scale network, is equivalent to a random routing strategy and that its transport capacity scales as $Θ(\sqrt{N})$ bit-meters per second, thus retrieving the scaling law that Gupta and Kumar (2000) obtained as the limit for single-route large-scale wireless networks.
△ Less
Submitted 16 January, 2006;
originally announced January 2006.
