-
Upper bounds for integrated information
Authors:
Alireza Zaeemzadeh,
Giulio Tononi
Abstract:
Originally developed as a theory of consciousness, integrated information theory provides a mathematical framework to quantify the causal irreducibility of systems and subsets of units in the system. Specifically, mechanism integrated information quantifies how much of the causal powers of a subset of units in a state, also referred to as a mechanism, cannot be accounted for by its parts. If the c…
▽ More
Originally developed as a theory of consciousness, integrated information theory provides a mathematical framework to quantify the causal irreducibility of systems and subsets of units in the system. Specifically, mechanism integrated information quantifies how much of the causal powers of a subset of units in a state, also referred to as a mechanism, cannot be accounted for by its parts. If the causal powers of the mechanism can be fully explained by its parts, it is reducible and its integrated information is zero. Here, we study the upper bound of this measure and how it is achieved. We study mechanisms in isolation, groups of mechanisms, and groups of causal relations among mechanisms. We put forward new theoretical results that show mechanisms that share parts with each other cannot all achieve their maximum. We also introduce techniques to design systems that can maximize the integrated information of a subset of their mechanisms or relations. Our results can potentially be used to exploit the symmetries and constraints to reduce the computations significantly and to compare different connectivity profiles in terms of their maximal achievable integrated information.
△ Less
Submitted 21 April, 2024; v1 submitted 16 May, 2023;
originally announced May 2023.
-
Integrated information theory (IIT) 4.0: Formulating the properties of phenomenal existence in physical terms
Authors:
Larissa Albantakis,
Leonardo Barbosa,
Graham Findlay,
Matteo Grasso,
Andrew M Haun,
William Marshall,
William GP Mayner,
Alireza Zaeemzadeh,
Melanie Boly,
Bjørn E Juel,
Shuntaro Sasai,
Keiko Fujii,
Isaac David,
Jeremiah Hendren,
Jonathan P Lang,
Giulio Tononi
Abstract:
This paper presents Integrated Information Theory (IIT) 4.0. IIT aims to account for the properties of experience in physical (operational) terms. It identifies the essential properties of experience (axioms), infers the necessary and sufficient properties that its substrate must satisfy (postulates), and expresses them in mathematical terms. In principle, the postulates can be applied to any syst…
▽ More
This paper presents Integrated Information Theory (IIT) 4.0. IIT aims to account for the properties of experience in physical (operational) terms. It identifies the essential properties of experience (axioms), infers the necessary and sufficient properties that its substrate must satisfy (postulates), and expresses them in mathematical terms. In principle, the postulates can be applied to any system of units in a state to determine whether it is conscious, to what degree, and in what way. IIT offers a parsimonious explanation of empirical evidence, makes testable predictions, and permits inferences and extrapolations. IIT 4.0 incorporates several developments of the past ten years, including a more accurate translation of axioms into postulates and mathematical expressions, the introduction of a unique measure of intrinsic information that is consistent with the postulates, and an explicit assessment of causal relations. By fully unfolding a system's irreducible cause-effect power, the distinctions and relations specified by a substrate can account for the quality of experience.
△ Less
Submitted 30 December, 2022;
originally announced December 2022.
-
System Integrated Information
Authors:
William Marshall,
Matteo Grasso,
William GP Mayner,
Alireza Zaeemzadeh,
Leonardo S Barbosa,
Erick Chastain,
Graham Findlay,
Shuntaro Sasai,
Larissa Albantakis,
Giulio Tononi
Abstract:
Integrated information theory (IIT) starts from consciousness itself and identifies a set of properties (axioms) that are true of every conceivable experience. The axioms are translated into a set of postulates about the substrate of consciousness (called a complex), which are then used to formulate a mathematical framework for assessing both the quality and quantity of experience. The explanatory…
▽ More
Integrated information theory (IIT) starts from consciousness itself and identifies a set of properties (axioms) that are true of every conceivable experience. The axioms are translated into a set of postulates about the substrate of consciousness (called a complex), which are then used to formulate a mathematical framework for assessing both the quality and quantity of experience. The explanatory identity proposed by IIT is that an experience is identical to the cause-effect structure unfolded from a maximally irreducible substrate (a $Φ$-structure). In this work we introduce a definition for the integrated information of a system ($\varphi_s$) that is based on the existence, intrinsicality, information, and integration postulates of IIT. We explore how notions of determinism, degeneracy, and fault lines in the connectivity impact system integrated information. We then demonstrate how the proposed measure identifies complexes as systems whose $\varphi_s$ is greater than the $\varphi_s$ of any overlap** candidate systems.
△ Less
Submitted 29 December, 2022;
originally announced December 2022.
-
RL-NCS: Reinforcement learning based data-driven approach for nonuniform compressed sensing
Authors:
Nazmul Karim,
Alireza Zaeemzadeh,
Nazanin Rahnavard
Abstract:
A reinforcement-learning-based non-uniform compressed sensing (NCS) framework for time-varying signals is introduced. The proposed scheme, referred to as RL-NCS, aims to boost the performance of signal recovery through an optimal and adaptive distribution of sensing energy among two groups of coefficients of the signal, referred to as the region of interest (ROI) coefficients and non-ROI coefficie…
▽ More
A reinforcement-learning-based non-uniform compressed sensing (NCS) framework for time-varying signals is introduced. The proposed scheme, referred to as RL-NCS, aims to boost the performance of signal recovery through an optimal and adaptive distribution of sensing energy among two groups of coefficients of the signal, referred to as the region of interest (ROI) coefficients and non-ROI coefficients. The coefficients in ROI usually have greater importance and need to be reconstructed with higher accuracy compared to non-ROI coefficients. In order to accomplish this task, the ROI is predicted at each time step using two specific approaches. One of these approaches incorporates a long short-term memory (LSTM) network for the prediction. The other approach employs the previous ROI information for predicting the next step ROI. Using the exploration-exploitation technique, a Q-network learns to choose the best approach for designing the measurement matrix. Furthermore, a joint loss function is introduced for the efficient training of the Q-network as well as the LSTM network. The result indicates a significant performance gain for our proposed method, even for rapidly varying signals and a reduced number of measurements.
△ Less
Submitted 2 July, 2021;
originally announced July 2021.
-
AQuRate: MRAM-based Stochastic Oscillator for Adaptive Quantization Rate Sampling of Sparse Signals
Authors:
Soheil Salehi,
Ramtin Zand,
Alireza Zaeemzadeh,
Nazanin Rahnavard,
Ronald F. DeMara
Abstract:
Recently, the promising aspects of compressive sensing have inspired new circuit-level approaches for their efficient realization within the literature. However, most of these recent advances involving novel sampling techniques have been proposed without considering hardware and signal constraints. Additionally, traditional hardware designs for generating non-uniform sampling clock incur large are…
▽ More
Recently, the promising aspects of compressive sensing have inspired new circuit-level approaches for their efficient realization within the literature. However, most of these recent advances involving novel sampling techniques have been proposed without considering hardware and signal constraints. Additionally, traditional hardware designs for generating non-uniform sampling clock incur large area overhead and power dissipation. Herein, we propose a novel non-uniform clock generator called Adaptive Quantization Rate (AQR) generator using Magnetic Random Access Memory (MRAM)-based stochastic oscillator devices. Our proposed AQR generator provides ~25-fold reduction in area, on average, while offering ~6-fold reduced power dissipation, on average, compared to the state-of-the-art non-uniform clock generators.
△ Less
Submitted 12 March, 2019; v1 submitted 3 March, 2019;
originally announced March 2019.
-
Iterative Projection and Matching: Finding Structure-preserving Representatives and Its Application to Computer Vision
Authors:
Mohsen Joneidi,
Alireza Zaeemzadeh,
Nazanin Rahnavard,
Mubarak Shah
Abstract:
The goal of data selection is to capture the most structural information from a set of data. This paper presents a fast and accurate data selection method, in which the selected samples are optimized to span the subspace of all data. We propose a new selection algorithm, referred to as iterative projection and matching (IPM), with linear complexity w.r.t. the number of data, and without any parame…
▽ More
The goal of data selection is to capture the most structural information from a set of data. This paper presents a fast and accurate data selection method, in which the selected samples are optimized to span the subspace of all data. We propose a new selection algorithm, referred to as iterative projection and matching (IPM), with linear complexity w.r.t. the number of data, and without any parameter to be tuned. In our algorithm, at each iteration, the maximum information from the structure of the data is captured by one selected sample, and the captured information is neglected in the next iterations by projection on the null-space of previously selected samples. The computational efficiency and the selection accuracy of our proposed algorithm outperform those of the conventional methods. Furthermore, the superiority of the proposed algorithm is shown on active learning for video action recognition dataset on UCF-101; learning using representatives on ImageNet; training a generative adversarial network (GAN) to generate multi-view images from a single-view input on CMU Multi-PIE dataset; and video summarization on UTE Egocentric dataset.
△ Less
Submitted 29 November, 2018;
originally announced November 2018.
-
Norm-Preservation: Why Residual Networks Can Become Extremely Deep?
Authors:
Alireza Zaeemzadeh,
Nazanin Rahnavard,
Mubarak Shah
Abstract:
Augmenting neural networks with skip connections, as introduced in the so-called ResNet architecture, surprised the community by enabling the training of networks of more than 1,000 layers with significant performance gains. This paper deciphers ResNet by analyzing the effect of skip connections, and puts forward new theoretical results on the advantages of identity skip connections in neural netw…
▽ More
Augmenting neural networks with skip connections, as introduced in the so-called ResNet architecture, surprised the community by enabling the training of networks of more than 1,000 layers with significant performance gains. This paper deciphers ResNet by analyzing the effect of skip connections, and puts forward new theoretical results on the advantages of identity skip connections in neural networks. We prove that the skip connections in the residual blocks facilitate preserving the norm of the gradient, and lead to stable back-propagation, which is desirable from optimization perspective. We also show that, perhaps surprisingly, as more residual blocks are stacked, the norm-preservation of the network is enhanced. Our theoretical arguments are supported by extensive empirical evidence. Can we push for extra norm-preservation? We answer this question by proposing an efficient method to regularize the singular values of the convolution operator and making the ResNet's transition layers extra norm-preserving. Our numerical investigations demonstrate that the learning dynamics and the classification performance of ResNet can be improved by making it even more norm preserving. Our results and the introduced modification for ResNet, referred to as Procrustes ResNets, can be used as a guide for training deeper networks and can also inspire new deeper architectures.
△ Less
Submitted 22 April, 2020; v1 submitted 18 May, 2018;
originally announced May 2018.
-
Dynamic Sensor Selection for Reliable Spectrum Sensing via E-Optimal Criterion
Authors:
Mohsen Joneidi,
Alireza Zaeemzadeh,
Nazanin Rahnavard
Abstract:
Reliable and efficient spectrum sensing through dynamic selection of a subset of spectrum sensors is studied. The problem of selecting K sensor measurements from a set of M potential sensors is considered where K << M. In addition, K may be less than the dimension of the unknown variables of estimation. Through sensor selection, we reduce the problem to an under-determined system of equations with…
▽ More
Reliable and efficient spectrum sensing through dynamic selection of a subset of spectrum sensors is studied. The problem of selecting K sensor measurements from a set of M potential sensors is considered where K << M. In addition, K may be less than the dimension of the unknown variables of estimation. Through sensor selection, we reduce the problem to an under-determined system of equations with potentially infinite number of solutions. However, the sparsity of the underlying data facilitates limiting the set of solutions to a unique solution. Sparsity enables employing the emerging compressive sensing technique, where the compressed measurements are selected from a large number of potential sensors. This paper suggests selecting sensors in a way that the reduced system of equations constructs a well-conditioned measurement matrix. Our criterion for sensor selection is based on E-optimalily, which is highly related to the restricted isometry property that provides some guarantees for sparse solution obtained by L1 minimization. Moreover, the proposed framework exploits a feedback mechanism to evolve the selected sensors dynamically over time. The evolution aims to maximize the reliability of the sensed spectrum.
△ Less
Submitted 15 August, 2018; v1 submitted 14 February, 2018;
originally announced February 2018.
-
Robust Target Localization Based on Squared Range Iterative Reweighted Least Squares
Authors:
Alireza Zaeemzadeh,
Mohsen Joneidi,
Behzad Shahrasbi,
Nazanin Rahnavard
Abstract:
In this paper, the problem of target localization in the presence of outlying sensors is tackled. This problem is important in practice because in many real-world applications the sensors might report irrelevant data unintentionally or maliciously. The problem is formulated by applying robust statistics techniques on squared range measurements and two different approaches to solve the problem are…
▽ More
In this paper, the problem of target localization in the presence of outlying sensors is tackled. This problem is important in practice because in many real-world applications the sensors might report irrelevant data unintentionally or maliciously. The problem is formulated by applying robust statistics techniques on squared range measurements and two different approaches to solve the problem are proposed. The first approach is computationally efficient; however, only the objective convergence is guaranteed theoretically. On the other hand, the whole-sequence convergence of the second approach is established. To enjoy the benefit of both approaches, they are integrated to develop a hybrid algorithm that offers computational efficiency and theoretical guarantees. The algorithms are evaluated for different simulated and real-world scenarios. The numerical results show that the proposed methods meet the Cr'amer-Rao lower bound (CRLB) for a sufficiently large number of measurements. When the number of the measurements is small, the proposed position estimator does not achieve CRLB though it still outperforms several existing localization methods.
△ Less
Submitted 14 February, 2018;
originally announced February 2018.
-
Adaptive Non-uniform Compressive Sampling for Time-varying Signals
Authors:
Alireza Zaeemzadeh,
Mohsen Joneidi,
Nazanin Rahnavard
Abstract:
In this paper, adaptive non-uniform compressive sampling (ANCS) of time-varying signals, which are sparse in a proper basis, is introduced. ANCS employs the measurements of previous time steps to distribute the sensing energy among coefficients more intelligently. To this aim, a Bayesian inference method is proposed that does not require any prior knowledge of importance levels of coefficients or…
▽ More
In this paper, adaptive non-uniform compressive sampling (ANCS) of time-varying signals, which are sparse in a proper basis, is introduced. ANCS employs the measurements of previous time steps to distribute the sensing energy among coefficients more intelligently. To this aim, a Bayesian inference method is proposed that does not require any prior knowledge of importance levels of coefficients or sparsity of the signal. Our numerical simulations show that ANCS is able to achieve the desired non-uniform recovery of the signal. Moreover, if the signal is sparse in canonical basis, ANCS can reduce the number of required measurements significantly.
△ Less
Submitted 9 March, 2017;
originally announced March 2017.
-
Missing Spectrum-Data Recovery in Cognitive Radio Networks Using Piecewise Constant Nonnegative Matrix Factorization
Authors:
Alireza Zaeemzadeh,
Mohsen Joneidi,
Behzad Shahrasbi,
Nazanin Rahnavard
Abstract:
In this paper, we propose a missing spectrum data recovery technique for cognitive radio (CR) networks using Nonnegative Matrix Factorization (NMF). It is shown that the spectrum measurements collected from secondary users (SUs) can be factorized as product of a channel gain matrix times an activation matrix. Then, an NMF method with piecewise constant activation coefficients is introduced to anal…
▽ More
In this paper, we propose a missing spectrum data recovery technique for cognitive radio (CR) networks using Nonnegative Matrix Factorization (NMF). It is shown that the spectrum measurements collected from secondary users (SUs) can be factorized as product of a channel gain matrix times an activation matrix. Then, an NMF method with piecewise constant activation coefficients is introduced to analyze the measurements and estimate the missing spectrum data. The proposed optimization problem is solved by a Majorization-Minimization technique. The numerical simulation verifies that the proposed technique is able to accurately estimate the missing spectrum data in the presence of noise and fading.
△ Less
Submitted 28 August, 2015;
originally announced August 2015.
-
Matrix Coherency Graph: A Tool for Improving Sparse Coding Performance
Authors:
Mohsen Joneidi,
Mahdi Barzegar Khalilsarai,
Alireza Zaeemzadeh,
Nazanin Rahnavard
Abstract:
Exact recovery of a sparse solution for an underdetermined system of linear equations implies full search among all possible subsets of the dictionary, which is computationally intractable, while l1 minimization will do the job when a Restricted Isometry Property holds for the dictionary. Yet, practical sparse recovery algorithms may fail to recover the vector of coefficients even when the diction…
▽ More
Exact recovery of a sparse solution for an underdetermined system of linear equations implies full search among all possible subsets of the dictionary, which is computationally intractable, while l1 minimization will do the job when a Restricted Isometry Property holds for the dictionary. Yet, practical sparse recovery algorithms may fail to recover the vector of coefficients even when the dictionary deviates from the RIP only slightly. To enjoy l1 minimization guarantees in a wider sense, a method based on a combination of full-search and l1 minimization is presented. The idea is based on partitioning the dictionary into atoms which are in some sense well-conditioned and those which are ill-conditioned. Inspired by that, a matrix coherency graph is introduced which is a tool extracted by the structure of the dictionary. This tool can be used for decreasing the greediness of sparse coding algorithms so that recovery will be more reliable. We have modified the IRLS algorithm by applying the proposed method on it and simulation results show that the modified version performs quite better than the original algorithm.
△ Less
Submitted 30 November, 2014;
originally announced December 2014.
