Showing 1–36 of 36 results for author: Salehkaleybar, S

Causal Effect Identification in LiNGAM Models with Latent Confounders

Authors: Daniele Tramontano, Yaroslav Kivva, Saber Salehkaleybar, Mathias Drton, Negar Kiyavash

Abstract: We study the generic identifiability of causal effects in linear non-Gaussian acyclic models (LiNGAM) with latent variables. We consider the problem in two main settings: When the causal graph is known a priori, and when it is unknown. In both settings, we provide a complete graphical characterization of the identifiable direct or total causal effects among observed variables. Moreover, we propose… ▽ More We study the generic identifiability of causal effects in linear non-Gaussian acyclic models (LiNGAM) with latent variables. We consider the problem in two main settings: When the causal graph is known a priori, and when it is unknown. In both settings, we provide a complete graphical characterization of the identifiable direct or total causal effects among observed variables. Moreover, we propose efficient algorithms to certify the graphical conditions. Finally, we propose an adaptation of the reconstruction independent component analysis (RICA) algorithm that estimates the causal effects from the observational data given the causal graph. Experimental results show the effectiveness of the proposed method in estimating the causal effects. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Comments: Accepted at International Conference on Machine Learning (ICML) 2024

Soft Preference Optimization: Aligning Language Models to Expert Distributions

Authors: Arsalan Sharifnassab, Sina Ghiassian, Saber Salehkaleybar, Surya Kanoria, Dale Schuurmans

Abstract: We propose Soft Preference Optimization (SPO), a method for aligning generative models, such as Large Language Models (LLMs), with human preferences, without the need for a reward model. SPO optimizes model outputs directly over a preference dataset through a natural loss function that integrates preference loss with a regularization term across the model's entire output distribution rather than l… ▽ More We propose Soft Preference Optimization (SPO), a method for aligning generative models, such as Large Language Models (LLMs), with human preferences, without the need for a reward model. SPO optimizes model outputs directly over a preference dataset through a natural loss function that integrates preference loss with a regularization term across the model's entire output distribution rather than limiting it to the preference dataset. Although SPO does not require the assumption of an existing underlying reward model, we demonstrate that, under the Bradley-Terry (BT) model assumption, it converges to a softmax of scaled rewards, with the distribution's "softness" adjustable via the softmax exponent, an algorithm parameter. We showcase SPO's methodology, its theoretical foundation, and its comparative advantages in simplicity, computational efficiency, and alignment precision. △ Less

Submitted 27 May, 2024; v1 submitted 30 April, 2024; originally announced May 2024.

MetaOptimize: A Framework for Optimizing Step Sizes and Other Meta-parameters

Authors: Arsalan Sharifnassab, Saber Salehkaleybar, Richard Sutton

Abstract: This paper addresses the challenge of optimizing meta-parameters (i.e., hyperparameters) in machine learning algorithms, a critical factor influencing training efficiency and model performance. Moving away from the computationally expensive traditional meta-parameter search methods, we introduce MetaOptimize framework that dynamically adjusts meta-parameters, particularly step sizes (also known as… ▽ More This paper addresses the challenge of optimizing meta-parameters (i.e., hyperparameters) in machine learning algorithms, a critical factor influencing training efficiency and model performance. Moving away from the computationally expensive traditional meta-parameter search methods, we introduce MetaOptimize framework that dynamically adjusts meta-parameters, particularly step sizes (also known as learning rates), during training. More specifically, MetaOptimize can wrap around any first-order optimization algorithm, tuning step sizes on the fly to minimize a specific form of regret that accounts for long-term effect of step sizes on training, through a discounted sum of future losses. We also introduce low complexity variants of MetaOptimize that, in conjunction with its adaptability to multiple optimization algorithms, demonstrate performance competitive to those of best hand-crafted learning rate schedules across various machine learning applications. △ Less

Submitted 27 May, 2024; v1 submitted 3 February, 2024; originally announced February 2024.

Learning Unknown Intervention Targets in Structural Causal Models from Heterogeneous Data

Authors: Yuqin Yang, Saber Salehkaleybar, Negar Kiyavash

Abstract: We study the problem of identifying the unknown intervention targets in structural causal models where we have access to heterogeneous data collected from multiple environments. The unknown intervention targets are the set of endogenous variables whose corresponding exogenous noises change across the environments. We propose a two-phase approach which in the first phase recovers the exogenous nois… ▽ More We study the problem of identifying the unknown intervention targets in structural causal models where we have access to heterogeneous data collected from multiple environments. The unknown intervention targets are the set of endogenous variables whose corresponding exogenous noises change across the environments. We propose a two-phase approach which in the first phase recovers the exogenous noises corresponding to unknown intervention targets whose distributions have changed across environments. In the second phase, the recovered noises are matched with the corresponding endogenous variables. For the recovery phase, we provide sufficient conditions for learning these exogenous noises up to some component-wise invertible transformation. For the matching phase, under the causal sufficiency assumption, we show that the proposed method uniquely identifies the intervention targets. In the presence of latent confounders, the intervention targets among the observed variables cannot be determined uniquely. We provide a candidate intervention target set which is a superset of the true intervention targets. Our approach improves upon the state of the art as the returned candidate set is always a subset of the target set returned by previous work. Moreover, we do not require restrictive assumptions such as linearity of the causal model or performing invariance tests to learn whether a distribution is changing across environments which could be highly sample inefficient. Our experimental results show the effectiveness of our proposed algorithm in practice. △ Less

Submitted 9 March, 2024; v1 submitted 10 December, 2023; originally announced December 2023.

Comments: Accepted at 27th International Conference on Artificial Intelligence and Statistics (AISTATS 2024)

Efficiently Esca** Saddle Points for Non-Convex Policy Optimization

Authors: Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash, Niao He, Matthias Grossglauser

Abstract: Policy gradient (PG) is widely used in reinforcement learning due to its scalability and good performance. In recent years, several variance-reduced PG methods have been proposed with a theoretical guarantee of converging to an approximate first-order stationary point (FOSP) with the sample complexity of $O(ε^{-3})$. However, FOSPs could be bad local optima or saddle points. Moreover, these algori… ▽ More Policy gradient (PG) is widely used in reinforcement learning due to its scalability and good performance. In recent years, several variance-reduced PG methods have been proposed with a theoretical guarantee of converging to an approximate first-order stationary point (FOSP) with the sample complexity of $O(ε^{-3})$. However, FOSPs could be bad local optima or saddle points. Moreover, these algorithms often use importance sampling (IS) weights which could impair the statistical effectiveness of variance reduction. In this paper, we propose a variance-reduced second-order method that uses second-order information in the form of Hessian vector products (HVP) and converges to an approximate second-order stationary point (SOSP) with sample complexity of $\tilde{O}(ε^{-3})$. This rate improves the best-known sample complexity for achieving approximate SOSPs by a factor of $O(ε^{-0.5})$. Moreover, the proposed variance reduction technique bypasses IS weights by using HVP terms. Our experimental results show that the proposed algorithm outperforms the state of the art and is more robust to changes in random seeds. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Comments: arXiv admin note: text overlap with arXiv:2205.08253

MSC Class: ACM-class:I.2.6

A Cross-Moment Approach for Causal Effect Estimation

Authors: Yaroslav Kivva, Saber Salehkaleybar, Negar Kiyavash

Abstract: We consider the problem of estimating the causal effect of a treatment on an outcome in linear structural causal models (SCM) with latent confounders when we have access to a single proxy variable. Several methods (such as difference-in-difference (DiD) estimator or negative outcome control) have been proposed in this setting in the literature. However, these approaches require either restrictive… ▽ More We consider the problem of estimating the causal effect of a treatment on an outcome in linear structural causal models (SCM) with latent confounders when we have access to a single proxy variable. Several methods (such as difference-in-difference (DiD) estimator or negative outcome control) have been proposed in this setting in the literature. However, these approaches require either restrictive assumptions on the data generating model or having access to at least two proxy variables. We propose a method to estimate the causal effect using cross moments between the treatment, the outcome, and the proxy variable. In particular, we show that the causal effect can be identified with simple arithmetic operations on the cross moments if the latent confounder in linear SCM is non-Gaussian. In this setting, DiD estimator provides an unbiased estimate only in the special case where the latent confounder has exactly the same direct causal effects on the outcomes in the pre-treatment and post-treatment phases. This translates to the common trend assumption in DiD, which we effectively relax. Additionally, we provide an impossibility result that shows the causal effect cannot be identified if the observational distribution over the treatment, the outcome, and the proxy is jointly Gaussian. Our experiments on both synthetic and real-world datasets showcase the effectiveness of the proposed approach in estimating the causal effect. △ Less

Submitted 9 June, 2023; originally announced June 2023.

Fast Causal Orientation Learning in Directed Acyclic Graphs

Authors: Ramin Safaeian, Saber Salehkaleybar, Mahmoud Tabandeh

Abstract: Causal relationships among a set of variables are commonly represented by a directed acyclic graph. The orientations of some edges in the causal DAG can be discovered from observational/interventional data. Further edges can be oriented by iteratively applying so-called Meek rules. Inferring edges' orientations from some previously oriented edges, which we call Causal Orientation Learning (COL), i… ▽ More Causal relationships among a set of variables are commonly represented by a directed acyclic graph. The orientations of some edges in the causal DAG can be discovered from observational/interventional data. Further edges can be oriented by iteratively applying so-called Meek rules. Inferring edges' orientations from some previously oriented edges, which we call Causal Orientation Learning (COL), is a common problem in various causal discovery tasks. In these tasks, it is often required to solve multiple COL problems and therefore applying Meek rules could be time-consuming. Motivated by Meek rules, we introduce Meek functions that can be utilized in solving COL problems. In particular, we show that these functions have some desirable properties, enabling us to speed up the process of applying Meek rules. In particular, we propose a dynamic programming (DP) based method to apply Meek functions. Moreover, based on the proposed DP method, we present a lower bound on the number of edges that can be oriented as a result of intervention. We also propose a method to check whether some oriented edges belong to a causal DAG. Experimental results show that the proposed methods can outperform previous work in several causal discovery tasks in terms of running-time. △ Less

Submitted 27 May, 2022; originally announced May 2022.

Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function

Authors: Saeed Masiha, Saber Salehkaleybar, Niao He, Negar Kiyavash, Patrick Thiran

Abstract: We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with $1\leα\le2$ which holds in a wide range of applications in machine learning and signal processing. This condition ensures that any first-order stationary point is a global optimum. We prove that the total sample complexity of SCRN in achieving $ε$-global optimu… ▽ More We study the performance of Stochastic Cubic Regularized Newton (SCRN) on a class of functions satisfying gradient dominance property with $1\leα\le2$ which holds in a wide range of applications in machine learning and signal processing. This condition ensures that any first-order stationary point is a global optimum. We prove that the total sample complexity of SCRN in achieving $ε$-global optimum is $\mathcal{O}(ε^{-7/(2α)+1})$ for $1\leα< 3/2$ and $\mathcal{\tilde{O}}(ε^{-2/(α)})$ for $3/2\leα\le 2$. SCRN improves the best-known sample complexity of stochastic gradient descent. Even under a weak version of gradient dominance property, which is applicable to policy-based reinforcement learning (RL), SCRN achieves the same improvement over stochastic policy gradient methods. Additionally, we show that the average sample complexity of SCRN can be reduced to ${\mathcal{O}}(ε^{-2})$ for $α=1$ using a variance reduction method with time-varying batch sizes. Experimental results in various RL settings showcase the remarkable performance of SCRN compared to first-order methods. △ Less

Submitted 20 January, 2023; v1 submitted 25 May, 2022; originally announced May 2022.

A Unified Experiment Design Approach for Cyclic and Acyclic Causal Models

Authors: Ehsan Mokhtarian, Saber Salehkaleybar, AmirEmad Ghassami, Negar Kiyavash

Abstract: We study experiment design for unique identification of the causal graph of a simple SCM, where the graph may contain cycles. The presence of cycles in the structure introduces major challenges for experiment design as, unlike acyclic graphs, learning the skeleton of causal graphs with cycles may not be possible from merely the observational distribution. Furthermore, intervening on a variable in… ▽ More We study experiment design for unique identification of the causal graph of a simple SCM, where the graph may contain cycles. The presence of cycles in the structure introduces major challenges for experiment design as, unlike acyclic graphs, learning the skeleton of causal graphs with cycles may not be possible from merely the observational distribution. Furthermore, intervening on a variable in such graphs does not necessarily lead to orienting all the edges incident to it. In this paper, we propose an experiment design approach that can learn both cyclic and acyclic graphs and hence, unifies the task of experiment design for both types of graphs. We provide a lower bound on the number of experiments required to guarantee the unique identification of the causal graph in the worst case, showing that the proposed approach is order-optimal in terms of the number of experiments up to an additive logarithmic term. Moreover, we extend our result to the setting where the size of each experiment is bounded by a constant. For this case, we show that our approach is optimal in terms of the size of the largest experiment required for uniquely identifying the causal graph in the worst case. △ Less

Submitted 13 December, 2023; v1 submitted 20 May, 2022; originally announced May 2022.

Comments: 31 pages, 6 figures, 1 table, accepted in JMLR

Momentum-Based Policy Gradient with Second-Order Information

Authors: Saber Salehkaleybar, Sadegh Khorasani, Negar Kiyavash, Niao He, Patrick Thiran

Abstract: Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance-reduced policy-gradient method, called SHARP, which incorporates second-order information into stochastic gradient descent (SGD) using momentum with a time-varying learn… ▽ More Variance-reduced gradient estimators for policy gradient methods have been one of the main focus of research in the reinforcement learning in recent years as they allow acceleration of the estimation process. We propose a variance-reduced policy-gradient method, called SHARP, which incorporates second-order information into stochastic gradient descent (SGD) using momentum with a time-varying learning rate. SHARP algorithm is parameter-free, achieving $ε$-approximate first-order stationary point with $O(ε^{-3})$ number of trajectories, while using a batch size of $O(1)$ at each iteration. Unlike most previous work, our proposed algorithm does not require importance sampling which can compromise the advantage of variance reduction process. Moreover, the variance of estimation error decays with the fast rate of $O(1/t^{2/3})$ where $t$ is the number of iterations. Our extensive experimental evaluations show the effectiveness of the proposed algorithm on various control tasks and its advantage over the state of the art in practice. △ Less

Submitted 26 November, 2023; v1 submitted 17 May, 2022; originally announced May 2022.

Causal Imitative Model for Autonomous Driving

Authors: Mohammad Reza Samsami, Mohammadhossein Bahari, Saber Salehkaleybar, Alexandre Alahi

Abstract: Imitation learning is a powerful approach for learning autonomous driving policy by leveraging data from expert driver demonstrations. However, driving policies trained via imitation learning that neglect the causal structure of expert demonstrations yield two undesirable behaviors: inertia and collision. In this paper, we propose Causal Imitative Model (CIM) to address inertia and collision probl… ▽ More Imitation learning is a powerful approach for learning autonomous driving policy by leveraging data from expert driver demonstrations. However, driving policies trained via imitation learning that neglect the causal structure of expert demonstrations yield two undesirable behaviors: inertia and collision. In this paper, we propose Causal Imitative Model (CIM) to address inertia and collision problems. CIM explicitly discovers the causal model and utilizes it to train the policy. Specifically, CIM disentangles the input to a set of latent variables, selects the causal variables, and determines the next position by leveraging the selected variables. Our experiments show that our method outperforms previous work in terms of inertia and collision rates. Moreover, thanks to exploiting the causal structure, CIM shrinks the input dimension to only two, hence, can adapt to new environments in a few-shot setting. Code is available at https://github.com/vita-epfl/CIM. △ Less

Submitted 7 December, 2021; originally announced December 2021.

ParaLiNGAM: Parallel Causal Structure Learning for Linear non-Gaussian Acyclic Models

Authors: Amirhossein Shahbazinia, Saber Salehkaleybar, Matin Hashemi

Abstract: One of the key objectives in many fields in machine learning is to discover causal relationships among a set of variables from observational data. In linear non-Gaussian acyclic models (LiNGAM), it can be shown that the true underlying causal structure can be identified uniquely from merely observational data. DirectLiNGAM algorithm is a well-known solution to learn the true causal structure in hi… ▽ More One of the key objectives in many fields in machine learning is to discover causal relationships among a set of variables from observational data. In linear non-Gaussian acyclic models (LiNGAM), it can be shown that the true underlying causal structure can be identified uniquely from merely observational data. DirectLiNGAM algorithm is a well-known solution to learn the true causal structure in high dimensional setting. DirectLiNGAM algorithm executes in a sequence of iterations and it performs a set of comparisons between pairs of variables in each iteration. Unfortunately, the runtime of this algorithm grows significantly as the number of variables increases. In this paper, we propose a parallel algorithm, called ParaLiNGAM, to learn casual structures based on DirectLiNGAM algorithm. We propose a threshold mechanism that can reduce the number of comparisons remarkably compared with the sequential solution. Moreover, in order to further reduce runtime, we employ a messaging mechanism between workers and derive some mathematical formulations to simplify the execution of comparisons. We also present an implementation of ParaLiNGAM on GPU, considering hardware constraints. Experimental results on synthetic and real data show that the implementation of proposed algorithm on GPU can outperform DirectLiNGAM by a factor up to 4600 X. △ Less

Submitted 28 September, 2021; originally announced September 2021.

Comments: submitted for peer review

Order Optimal Bounds for One-Shot Federated Learning over non-Convex Loss Functions

Authors: Arsalan Sharifnassab, Saber Salehkaleybar, S. Jamaloddin Golestani

Abstract: We consider the problem of federated learning in a one-shot setting in which there are $m$ machines, each observing $n$ sample functions from an unknown distribution on non-convex loss functions. Let $F:[-1,1]^d\to\mathbb{R}$ be the expected loss function with respect to this unknown distribution. The goal is to find an estimate of the minimizer of $F$. Based on its observations, each machine gene… ▽ More We consider the problem of federated learning in a one-shot setting in which there are $m$ machines, each observing $n$ sample functions from an unknown distribution on non-convex loss functions. Let $F:[-1,1]^d\to\mathbb{R}$ be the expected loss function with respect to this unknown distribution. The goal is to find an estimate of the minimizer of $F$. Based on its observations, each machine generates a signal of bounded length $B$ and sends it to a server. The server collects signals of all machines and outputs an estimate of the minimizer of $F$. We show that the expected loss of any algorithm is lower bounded by $\max\big(1/(\sqrt{n}(mB)^{1/d}), 1/\sqrt{mn}\big)$, up to a logarithmic factor. We then prove that this lower bound is order optimal in $m$ and $n$ by presenting a distributed learning algorithm, called Multi-Resolution Estimator for Non-Convex loss function (MRE-NC), whose expected loss matches the lower bound for large $mn$ up to polylogarithmic factors. △ Less

Submitted 6 February, 2024; v1 submitted 19 August, 2021; originally announced August 2021.

Deep-Learning Based Blind Recognition of Channel Code Parameters over Candidate Sets under AWGN and Multi-Path Fading Conditions

Authors: Sepehr Dehdashtian, Matin Hashemi, Saber Salehkaleybar

Abstract: We consider the problem of recovering channel code parameters over a candidate set by merely analyzing the received encoded signals. We propose a deep learning-based solution that I) is capable of identifying the channel code parameters for any coding scheme (such as LDPC, Convolutional, Turbo, and Polar codes), II) is robust against channel impairments like multi-path fading, III) does not requir… ▽ More We consider the problem of recovering channel code parameters over a candidate set by merely analyzing the received encoded signals. We propose a deep learning-based solution that I) is capable of identifying the channel code parameters for any coding scheme (such as LDPC, Convolutional, Turbo, and Polar codes), II) is robust against channel impairments like multi-path fading, III) does not require any previous knowledge or estimation of channel state or signal-to-noise ratio (SNR), and IV) outperforms related works in terms of probability of detecting the correct code parameters. △ Less

Submitted 30 January, 2021; v1 submitted 16 September, 2020; originally announced September 2020.

Comments: accepted for publication in IEEE Wireless Communications Letters

gIM: GPU Accelerated RIS-based Influence Maximization Algorithm

Authors: Soheil Shahrouz, Saber Salehkaleybar, Matin Hashemi

Abstract: Given a social network modeled as a weighted graph $G$, the influence maximization problem seeks $k$ vertices to become initially influenced, to maximize the expected number of influenced nodes under a particular diffusion model. The influence maximization problem has been proven to be NP-hard, and most proposed solutions to the problem are approximate greedy algorithms, which can guarantee a tuna… ▽ More Given a social network modeled as a weighted graph $G$, the influence maximization problem seeks $k$ vertices to become initially influenced, to maximize the expected number of influenced nodes under a particular diffusion model. The influence maximization problem has been proven to be NP-hard, and most proposed solutions to the problem are approximate greedy algorithms, which can guarantee a tunable approximation ratio for their results with respect to the optimal solution. The state-of-the-art algorithms are based on Reverse Influence Sampling (RIS) technique, which can offer both computational efficiency and non-trivial $(1-\frac{1}{e}-ε)$-approximation ratio guarantee for any $ε>0$. RIS-based algorithms, despite their lower computational cost compared to other methods, still require long running times to solve the problem in large-scale graphs with low values of $ε$. In this paper, we present a novel and efficient parallel implementation of a RIS-based algorithm, namely IMM, on GPU. The proposed GPU-accelerated influence maximization algorithm, named gIM, can significantly reduce the running time on large-scale graphs with low values of $ε$. Furthermore, we show that gIM algorithm can solve other variations of the IM problem, only by applying minor modifications. Experimental results show that the proposed solution reduces the runtime by a factor up to $220 \times$. The source code of gIM is publicly available online. △ Less

Submitted 12 March, 2021; v1 submitted 15 September, 2020; originally announced September 2020.

Comments: Accepted for publication in IEEE Transactions on Parallel and Distributed Systems (TPDS)

Active Learning of Causal Structures with Deep Reinforcement Learning

Authors: Amir Amirinezhad, Saber Salehkaleybar, Matin Hashemi

Abstract: We study the problem of experiment design to learn causal structures from interventional data. We consider an active learning setting in which the experimenter decides to intervene on one of the variables in the system in each step and uses the results of the intervention to recover further causal relationships among the variables. The goal is to fully identify the causal structures with minimum n… ▽ More We study the problem of experiment design to learn causal structures from interventional data. We consider an active learning setting in which the experimenter decides to intervene on one of the variables in the system in each step and uses the results of the intervention to recover further causal relationships among the variables. The goal is to fully identify the causal structures with minimum number of interventions. We present the first deep reinforcement learning based solution for the problem of experiment design. In the proposed method, we embed input graphs to vectors using a graph neural network and feed them to another neural network which outputs a variable for performing intervention in each step. Both networks are trained jointly via a Q-iteration algorithm. Experimental results show that the proposed method achieves competitive performance in recovering causal structures with respect to previous works, while significantly reducing execution time in dense graphs. △ Less

Submitted 7 September, 2020; originally announced September 2020.

Comments: submitted for peer review

LazyIter: A Fast Algorithm for Counting Markov Equivalent DAGs and Designing Experiments

Authors: Ali AhmadiTeshnizi, Saber Salehkaleybar, Negar Kiyavash

Abstract: The causal relationships among a set of random variables are commonly represented by a Directed Acyclic Graph (DAG), where there is a directed edge from variable $X$ to variable $Y$ if $X$ is a direct cause of $Y$. From the purely observational data, the true causal graph can be identified up to a Markov Equivalence Class (MEC), which is a set of DAGs with the same conditional independencies betwe… ▽ More The causal relationships among a set of random variables are commonly represented by a Directed Acyclic Graph (DAG), where there is a directed edge from variable $X$ to variable $Y$ if $X$ is a direct cause of $Y$. From the purely observational data, the true causal graph can be identified up to a Markov Equivalence Class (MEC), which is a set of DAGs with the same conditional independencies between the variables. The size of an MEC is a measure of complexity for recovering the true causal graph by performing interventions. We propose a method for efficient iteration over possible MECs given intervention results. We utilize the proposed method for computing MEC sizes and experiment design in active and passive learning settings. Compared to previous work for computing the size of MEC, our proposed algorithm reduces the time complexity by a factor of $O(n)$ for sparse graphs where $n$ is the number of variables in the system. Additionally, integrating our approach with dynamic programming, we design an optimal algorithm for passive experiment design. Experimental results show that our proposed algorithms for both computing the size of MEC and experiment design outperform the state of the art. △ Less

Submitted 17 June, 2020; originally announced June 2020.

Comments: 11 pages, 2 figures, ICML

Order Optimal One-Shot Distributed Learning

Authors: Arsalan Sharifnassab, Saber Salehkaleybar, S. Jamaloddin Golestani

Abstract: We consider distributed statistical optimization in one-shot setting, where there are $m$ machines each observing $n$ i.i.d. samples. Based on its observed samples, each machine then sends an $O(\log(mn))$-length message to a server, at which a parameter minimizing an expected loss is to be estimated. We propose an algorithm called Multi-Resolution Estimator (MRE) whose expected error is no larger… ▽ More We consider distributed statistical optimization in one-shot setting, where there are $m$ machines each observing $n$ i.i.d. samples. Based on its observed samples, each machine then sends an $O(\log(mn))$-length message to a server, at which a parameter minimizing an expected loss is to be estimated. We propose an algorithm called Multi-Resolution Estimator (MRE) whose expected error is no larger than $\tilde{O}\big(m^{-{1}/{\max(d,2)}} n^{-1/2}\big)$, where $d$ is the dimension of the parameter space. This error bound meets existing lower bounds up to poly-logarithmic factors, and is thereby order optimal. The expected error of MRE, unlike existing algorithms, tends to zero as the number of machines ($m$) goes to infinity, even when the number of samples per machine ($n$) remains upper bounded by a constant. This property of the MRE algorithm makes it applicable in new machine learning paradigms where $m$ is much larger than $n$. △ Less

Submitted 2 November, 2019; originally announced November 2019.

Comments: arXiv admin note: substantial text overlap with arXiv:1905.04634

Journal ref: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

Distributed Voting in Beep Model

Authors: Benyamin Ghojogh, Saber Salehkaleybar

Abstract: We consider the problem of distributed multi-choice voting in a setting that each node can communicate with its neighbors merely by sending beep signals. Given its simplicity, the beep communication model is of practical importance in different applications such as system biology and wireless sensor networks. Yet, the distributed majority voting has not been resolved in this setting. In this paper… ▽ More We consider the problem of distributed multi-choice voting in a setting that each node can communicate with its neighbors merely by sending beep signals. Given its simplicity, the beep communication model is of practical importance in different applications such as system biology and wireless sensor networks. Yet, the distributed majority voting has not been resolved in this setting. In this paper, we propose two algorithms, named Distributed Voting with Beeps, to resolve this problem. In the first proposed algorithm, the adjacent nodes having the same value form a set called spot. Afterwards, the spots with majority value try to corrode the spots with non-majority values. The second proposed algorithm is based on pairwise interactions between nodes. The proposed algorithms have a termination detection procedure to check whether voting is achieved. We establish theoretical guarantees for the convergence of these algorithms. In particular, we show that the success probability of the first algorithm tends to one as the percentage of the initial votes in majority increases. For the second algorithm, we show that it returns the correct output with high probability. Our experiments show that the algorithms are fairly invariant to the network topology, initial distribution of values, and network size. △ Less

Submitted 7 August, 2020; v1 submitted 22 October, 2019; originally announced October 2019.

Comments: Published in Signal Processing journal, Elsevier, 2020

Journal ref: Signal Processing, Elsevier, vol. 177, 2020

Interventional Experiment Design for Causal Structure Learning

Authors: AmirEmad Ghassami, Saber Salehkaleybar, Negar Kiyavash

Abstract: It is known that from purely observational data, a causal DAG is identifiable only up to its Markov equivalence class, and for many ground truth DAGs, the direction of a large portion of the edges will be remained unidentified. The golden standard for learning the causal DAG beyond Markov equivalence is to perform a sequence of interventions in the system and use the data gathered from the interve… ▽ More It is known that from purely observational data, a causal DAG is identifiable only up to its Markov equivalence class, and for many ground truth DAGs, the direction of a large portion of the edges will be remained unidentified. The golden standard for learning the causal DAG beyond Markov equivalence is to perform a sequence of interventions in the system and use the data gathered from the interventional distributions. We consider a setup in which given a budget $k$, we design $k$ interventions non-adaptively. We cast the problem of finding the best intervention target set as an optimization problem which aims to maximize the number of edges whose directions are identified due to the performed interventions. First, we consider the case that the underlying causal structure is a tree. For this case, we propose an efficient exact algorithm for the worst-case gain setup, as well as an approximate algorithm for the average gain setup. We then show that the proposed approach for the average gain setup can be extended to the case of general causal structures. In this case, besides the design of interventions, calculating the objective function is also challenging. We propose an efficient exact calculator as well as two estimators for this task. We evaluate the proposed methods using synthetic as well as real data. △ Less

Submitted 12 October, 2019; originally announced October 2019.

Adversarial Orthogonal Regression: Two non-Linear Regressions for Causal Inference

Authors: M. Reza Heydari, Saber Salehkaleybar, Kun Zhang

Abstract: We propose two nonlinear regression methods, named Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors while putting no assumption on noise distribution. In both methods, two adversarial netwo… ▽ More We propose two nonlinear regression methods, named Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors while putting no assumption on noise distribution. In both methods, two adversarial networks are trained simultaneously where a regression network outputs predictions and a loss network that estimates mutual information (in AdOR) and KL-divergence (in AdOSE). These methods can be formulated as a minimax two-player game; at equilibrium, AdOR finds a deterministic map between inputs and output and estimates mutual information between residual and inputs, while AdOSE estimates a conditional probability distribution of output given inputs. The proposed methods can be used as subroutines to address several learning problems in causality, such as causal direction determination (or more generally, causal structure learning) and causal model estimation. Synthetic and real-world experiments demonstrate that the proposed methods have a remarkable performance with respect to previous solutions. △ Less

Submitted 10 September, 2019; originally announced September 2019.

Learning Linear Non-Gaussian Causal Models in the Presence of Latent Variables

Authors: Saber Salehkaleybar, AmirEmad Ghassami, Negar Kiyavash, Kun Zhang

Abstract: We consider the problem of learning causal models from observational data generated by linear non-Gaussian acyclic causal models with latent variables. Without considering the effect of latent variables, one usually infers wrong causal relationships among the observed variables. Under faithfulness assumption, we propose a method to check whether there exists a causal path between any two observed… ▽ More We consider the problem of learning causal models from observational data generated by linear non-Gaussian acyclic causal models with latent variables. Without considering the effect of latent variables, one usually infers wrong causal relationships among the observed variables. Under faithfulness assumption, we propose a method to check whether there exists a causal path between any two observed variables. From this information, we can obtain the causal order among them. The next question is then whether or not the causal effects can be uniquely identified as well. It can be shown that causal effects among observed variables cannot be identified uniquely even under the assumptions of faithfulness and non-Gaussianity of exogenous noises. However, we will propose an efficient method to identify the set of all possible causal effects that are compatible with the observational data. Furthermore, we present some structural conditions on the causal graph under which we can learn causal effects among observed variables uniquely. We also provide necessary and sufficient graphical conditions for unique identification of the number of variables in the system. Experiments on synthetic data and real-world data show the effectiveness of our proposed algorithm on learning causal models. △ Less

Submitted 11 August, 2019; originally announced August 2019.

Broadcast Distributed Voting Algorithm in Population Protocols

Authors: Hamidreza Bandealinaeini, Saber Salehkaleybar

Abstract: We consider the problem of multi-choice majority voting in a network of $n$ agents where each agent initially selects a choice from a set of $K$ possible choices. The agents try to infer the choice in majority merely by performing local interactions. Population protocols provide a framework for designing pairwise interactions between agents in order to perform tasks in a coordinated manner. In thi… ▽ More We consider the problem of multi-choice majority voting in a network of $n$ agents where each agent initially selects a choice from a set of $K$ possible choices. The agents try to infer the choice in majority merely by performing local interactions. Population protocols provide a framework for designing pairwise interactions between agents in order to perform tasks in a coordinated manner. In this paper, we propose ``Broadcasting Population Protocol" model as a counterpart model of conventional population protocols for the networks that each agent can send a message to all its neighbors simultaneously. We design two distributed algorithms for solving the multi-choice majority voting problem in the model of broadcasting population protocols. We prove the correctness of these algorithms and analyze their performance in terms of time and message complexities. Experiments show that the proposed algorithm improves both time and message complexities significantly with respect to previous algorithms proposed in conventional population protocols and they can be utilized in networks where messages can be transmitted to a subset of agents simultaneously such as wireless networks. △ Less

Submitted 16 July, 2019; originally announced July 2019.

Seedless Graph Matching via Tail of Degree Distribution for Correlated Erdos-Renyi Graphs

Authors: Mahdi Bozorg, Saber Salehkaleybar, Matin Hashemi

Abstract: The network alignment (or graph matching) problem refers to recovering the node-to-node correspondence between two correlated networks. In this paper, we propose a network alignment algorithm which works without using a seed set of pre-matched node pairs or any other auxiliary information (e.g., node or edge labels) as an input. The algorithm assigns structurally innovative features to nodes based… ▽ More The network alignment (or graph matching) problem refers to recovering the node-to-node correspondence between two correlated networks. In this paper, we propose a network alignment algorithm which works without using a seed set of pre-matched node pairs or any other auxiliary information (e.g., node or edge labels) as an input. The algorithm assigns structurally innovative features to nodes based on the tail of empirical degree distribution of their neighbor nodes. Then, it matches the nodes according to these features. We evaluate the performance of proposed algorithm on both synthetic and real networks. For synthetic networks, we generate Erdos-Renyi graphs in the regions of $Θ(\log(n)/n)$ and $Θ(\log^{2}(n)/n)$, where a previous work theoretically showed that recovering is feasible in sparse Erdos-Renyi graphs if and only if the probability of having an edge between a pair of nodes in one of the graphs and also between the corresponding nodes in the other graph is in the order of $Ω(\log(n)/n)$, where $n$ is the number of nodes. Experiments on both real and synthetic networks show that it outperforms previous works in terms of probability of correct matching. △ Less

Submitted 28 September, 2020; v1 submitted 15 July, 2019; originally announced July 2019.

Comments: submitted for peer review

One-Shot Federated Learning: Theoretical Limits and Algorithms to Achieve Them

Authors: Saber Salehkaleybar, Arsalan Sharifnassab, S. Jamaloddin Golestani

Abstract: We consider distributed statistical optimization in one-shot setting, where there are $m$ machines each observing $n$ i.i.d. samples. Based on its observed samples, each machine sends a $B$-bit-long message to a server. The server then collects messages from all machines, and estimates a parameter that minimizes an expected convex loss function. We investigate the impact of communication constrain… ▽ More We consider distributed statistical optimization in one-shot setting, where there are $m$ machines each observing $n$ i.i.d. samples. Based on its observed samples, each machine sends a $B$-bit-long message to a server. The server then collects messages from all machines, and estimates a parameter that minimizes an expected convex loss function. We investigate the impact of communication constraint, $B$, on the expected error and derive a tight lower bound on the error achievable by any algorithm. We then propose an estimator, which we call Multi-Resolution Estimator (MRE), whose expected error (when $B\ge\log mn$) meets the aforementioned lower bound up to poly-logarithmic factors, and is thereby order optimal. We also address the problem of learning under tiny communication budget, and present lower and upper error bounds when $B$ is a constant. The expected error of MRE, unlike existing algorithms, tends to zero as the number of machines ($m$) goes to infinity, even when the number of samples per machine ($n$) remains upper bounded by a constant. This property of the MRE algorithm makes it applicable in new machine learning paradigms where $m$ is much larger than $n$. △ Less

Submitted 30 December, 2019; v1 submitted 11 May, 2019; originally announced May 2019.

cuPC: CUDA-based Parallel PC Algorithm for Causal Structure Learning on GPU

Authors: Behrooz Zarebavani, Foad Jafarinejad, Matin Hashemi, Saber Salehkaleybar

Abstract: The main goal in many fields in the empirical sciences is to discover causal relationships among a set of variables from observational data. PC algorithm is one of the promising solutions to learn underlying causal structure by performing a number of conditional independence tests. In this paper, we propose a novel GPU-based parallel algorithm, called cuPC, to execute an order-independent version… ▽ More The main goal in many fields in the empirical sciences is to discover causal relationships among a set of variables from observational data. PC algorithm is one of the promising solutions to learn underlying causal structure by performing a number of conditional independence tests. In this paper, we propose a novel GPU-based parallel algorithm, called cuPC, to execute an order-independent version of PC. The proposed solution has two variants, cuPC-E and cuPC-S, which parallelize PC in two different ways for multivariate normal distribution. Experimental results show the scalability of the proposed algorithms with respect to the number of variables, the number of samples, and different graph densities. For instance, in one of the most challenging datasets, the runtime is reduced from more than 11 hours to about 4 seconds. On average, cuPC-E and cuPC-S achieve 500 X and 1300 X speedup, respectively, compared to serial implementation on CPU. The source code of cuPC is available online [1]. △ Less

Submitted 23 September, 2019; v1 submitted 20 December, 2018; originally announced December 2018.

Comments: Published in IEEE Transactions on Parallel and Distributed Systems (TPDS), 2019

Journal ref: IEEE Transactions on Parallel and Distributed Systems (TPDS), Vol. 31, No. 3, March 2020

Counting and Sampling from Markov Equivalent DAGs Using Clique Trees

Authors: AmirEmad Ghassami, Saber Salehkaleybar, Negar Kiyavash, Kun Zhang

Abstract: A directed acyclic graph (DAG) is the most common graphical model for representing causal relationships among a set of variables. When restricted to using only observational data, the structure of the ground truth DAG is identifiable only up to Markov equivalence, based on conditional independence relations among the variables. Therefore, the number of DAGs equivalent to the ground truth DAG is an… ▽ More A directed acyclic graph (DAG) is the most common graphical model for representing causal relationships among a set of variables. When restricted to using only observational data, the structure of the ground truth DAG is identifiable only up to Markov equivalence, based on conditional independence relations among the variables. Therefore, the number of DAGs equivalent to the ground truth DAG is an indicator of the causal complexity of the underlying structure--roughly speaking, it shows how many interventions or how much additional information is further needed to recover the underlying DAG. In this paper, we propose a new technique for counting the number of DAGs in a Markov equivalence class. Our approach is based on the clique tree representation of chordal graphs. We show that in the case of bounded degree graphs, the proposed algorithm is polynomial time. We further demonstrate that this technique can be utilized for uniform sampling from a Markov equivalence class, which provides a stochastic way to enumerate DAGs in the equivalence class and may be needed for finding the best DAG or for causal inference given the equivalence class as input. We also extend our counting and sampling method to the case where prior knowledge about the underlying DAG is available, and present applications of this extension in causal experiment design and estimating the causal effect of joint interventions. △ Less

Submitted 10 September, 2018; v1 submitted 4 February, 2018; originally announced February 2018.

Budgeted Experiment Design for Causal Structure Learning

Authors: AmirEmad Ghassami, Saber Salehkaleybar, Negar Kiyavash, Elias Bareinboim

Abstract: We study the problem of causal structure learning when the experimenter is limited to perform at most $k$ non-adaptive experiments of size $1$. We formulate the problem of finding the best intervention target set as an optimization problem, which aims to maximize the average number of edges whose directions are resolved. We prove that the corresponding objective function is submodular and a greedy… ▽ More We study the problem of causal structure learning when the experimenter is limited to perform at most $k$ non-adaptive experiments of size $1$. We formulate the problem of finding the best intervention target set as an optimization problem, which aims to maximize the average number of edges whose directions are resolved. We prove that the corresponding objective function is submodular and a greedy algorithm suffices to achieve $(1-\frac{1}{e})$-approximation of the optimal value. We further present an accelerated variant of the greedy algorithm, which can lead to orders of magnitude performance speedup. We validate our proposed approach on synthetic and real graphs. The results show that compared to the purely observational setting, our algorithm orients the majority of the edges through a considerably small number of interventions. △ Less

Submitted 29 July, 2018; v1 submitted 11 September, 2017; originally announced September 2017.

Journal ref: 35th International Conference on Machine Learning (ICML), PMLR 80:1719-1728, 2018

Learning Causal Structures Using Regression Invariance

Authors: AmirEmad Ghassami, Saber Salehkaleybar, Negar Kiyavash, Kun Zhang

Abstract: We study causal inference in a multi-environment setting, in which the functional relations for producing the variables from their direct causes remain the same across environments, while the distribution of exogenous noises may vary. We introduce the idea of using the invariance of the functional relations of the variables to their causes across a set of environments. We define a notion of comple… ▽ More We study causal inference in a multi-environment setting, in which the functional relations for producing the variables from their direct causes remain the same across environments, while the distribution of exogenous noises may vary. We introduce the idea of using the invariance of the functional relations of the variables to their causes across a set of environments. We define a notion of completeness for a causal inference algorithm in this setting and prove the existence of such algorithm by proposing the baseline algorithm. Additionally, we present an alternate algorithm that has significantly improved computational and sample complexity compared to the baseline algorithm. The experiment results show that the proposed algorithm outperforms the other existing algorithms. △ Less

Submitted 26 May, 2017; originally announced May 2017.

Distributed Voting/Ranking with Optimal Number of States per Node

Authors: Saber Salehkaleybar, Arsalan Sharif-Nassab, S. Jamaloddin Golestani

Abstract: Considering a network with $n$ nodes, where each node initially votes for one (or more) choices out of $K$ possible choices, we present a Distributed Multi-choice Voting/Ranking (DMVR) algorithm to determine either the choice with maximum vote (the voting problem) or to rank all the choices in terms of their acquired votes (the ranking problem). The algorithm consolidates node votes across the net… ▽ More Considering a network with $n$ nodes, where each node initially votes for one (or more) choices out of $K$ possible choices, we present a Distributed Multi-choice Voting/Ranking (DMVR) algorithm to determine either the choice with maximum vote (the voting problem) or to rank all the choices in terms of their acquired votes (the ranking problem). The algorithm consolidates node votes across the network by updating the states of interacting nodes using two key operations, the union and the intersection. The proposed algorithm is simple, independent from network size, and easily scalable in terms of the number of choices $K$, using only $K\times 2^{K-1}$ nodal states for voting, and $K\times K!$ nodal states for ranking. We prove the number of states to be optimal in the ranking case, this optimality is conjectured to also apply to the voting case. The time complexity of the algorithm is analyzed in complete graphs. We show that the time complexity for both ranking and voting is $O(\log(n))$ for given vote percentages, and is inversely proportional to the minimum of the vote percentage differences among various choices. △ Less

Submitted 26 March, 2017; originally announced March 2017.

Token-based Function Computation with Memory

Authors: Saber Salehkaleybar, S. Jamaloddin Golestani

Abstract: In distributed function computation, each node has an initial value and the goal is to compute a function of these values in a distributed manner. In this paper, we propose a novel token-based approach to compute a wide class of target functions to which we refer as "Token-based function Computation with Memory" (TCM) algorithm. In this approach, node values are attached to tokens and travel acros… ▽ More In distributed function computation, each node has an initial value and the goal is to compute a function of these values in a distributed manner. In this paper, we propose a novel token-based approach to compute a wide class of target functions to which we refer as "Token-based function Computation with Memory" (TCM) algorithm. In this approach, node values are attached to tokens and travel across the network. Each pair of travelling tokens would coalesce when they meet, forming a token with a new value as a function of the original token values. In contrast to the Coalescing Random Walk (CRW) algorithm, where token movement is governed by random walk, meeting of tokens in our scheme is accelerated by adopting a novel chasing mechanism. We proved that, compared to the CRW algorithm, the TCM algorithm results in a reduction of time complexity by a factor of at least $\sqrt{n/\log(n)}$ in Erdös-Renyi and complete graphs, and by a factor of $\log(n)/\log(\log(n))$ in torus networks. Simulation results show that there is at least a constant factor improvement in the message complexity of TCM algorithm in all considered topologies. Robustness of the CRW and TCM algorithms in the presence of node failure is analyzed. We show that their robustness can be improved by running multiple instances of the algorithms in parallel. △ Less

Submitted 26 March, 2017; originally announced March 2017.

Learning Vector Autoregressive Models with Latent Processes

Authors: Saber Salehkaleybar, Jalal Etesami, Negar Kiyavash, Kun Zhang

Abstract: We study the problem of learning the support of transition matrix between random processes in a Vector Autoregressive (VAR) model from samples when a subset of the processes are latent. It is well known that ignoring the effect of the latent processes may lead to very different estimates of the influences among observed processes, and we are concerned with identifying the influences among the obse… ▽ More We study the problem of learning the support of transition matrix between random processes in a Vector Autoregressive (VAR) model from samples when a subset of the processes are latent. It is well known that ignoring the effect of the latent processes may lead to very different estimates of the influences among observed processes, and we are concerned with identifying the influences among the observed processes, those between the latent ones, and those from the latent to the observed ones. We show that the support of transition matrix among the observed processes and lengths of all latent paths between any two observed processes can be identified successfully under some conditions on the VAR model. From the lengths of latent paths, we reconstruct the latent subgraph (representing the influences among the latent processes) with a minimum number of variables uniquely if its topology is a directed tree. Furthermore, we propose an algorithm that finds all possible minimal latent graphs under some conditions on the lengths of latent paths. Our results apply to both non-Gaussian and Gaussian cases, and experimental results on various synthetic and real-world datasets validate our theoretical results. △ Less

Submitted 9 November, 2017; v1 submitted 27 February, 2017; originally announced February 2017.

Optimal Experiment Design for Causal Discovery from Fixed Number of Experiments

Authors: AmirEmad Ghassami, Saber Salehkaleybar, Negar Kiyavash

Abstract: We study the problem of causal structure learning over a set of random variables when the experimenter is allowed to perform at most $M$ experiments in a non-adaptive manner. We consider the optimal learning strategy in terms of minimizing the portions of the structure that remains unknown given the limited number of experiments in both Bayesian and minimax setting. We characterize the theoretical… ▽ More We study the problem of causal structure learning over a set of random variables when the experimenter is allowed to perform at most $M$ experiments in a non-adaptive manner. We consider the optimal learning strategy in terms of minimizing the portions of the structure that remains unknown given the limited number of experiments in both Bayesian and minimax setting. We characterize the theoretical optimal solution and propose an algorithm, which designs the experiments efficiently in terms of time complexity. We show that for bounded degree graphs, in the minimax case and in the Bayesian case with uniform priors, our proposed algorithm is a $ρ$-approximation algorithm, where $ρ$ is independent of the order of the underlying graph. Simulations on both synthetic and real data show that the performance of our algorithm is very close to the optimal solution. △ Less

Submitted 27 February, 2017; originally announced February 2017.

Identifying Nonlinear 1-Step Causal Influences in Presence of Latent Variables

Authors: Saber Salehkaleybar, Jalal Etesami, Negar Kiyavash

Abstract: We propose an approach for learning the causal structure in stochastic dynamical systems with a $1$-step functional dependency in the presence of latent variables. We propose an information-theoretic approach that allows us to recover the causal relations among the observed variables as long as the latent variables evolve without exogenous noise. We further propose an efficient learning method bas… ▽ More We propose an approach for learning the causal structure in stochastic dynamical systems with a $1$-step functional dependency in the presence of latent variables. We propose an information-theoretic approach that allows us to recover the causal relations among the observed variables as long as the latent variables evolve without exogenous noise. We further propose an efficient learning method based on linear regression for the special sub-case when the dynamics are restricted to be linear. We validate the performance of our approach via numerical simulations. △ Less

Submitted 23 January, 2017; originally announced January 2017.

QoS-Aware Joint Policies in Cognitive Radio Networks

Authors: Saber Salehkaleybar, Seyyed Arash Majd, Mohammad Reza Pakravan

Abstract: One of the most challenging problems in Opportunistic Spectrum Access (OSA) is to design channel sensing-based protocol in multi secondary users (SUs) network. Quality of Service (QoS) requirements for SUs have significant implications on this protocol design. In this paper, we propose a new method to find joint policies for SUs which not only guarantees QoS requirements but also maximizes network… ▽ More One of the most challenging problems in Opportunistic Spectrum Access (OSA) is to design channel sensing-based protocol in multi secondary users (SUs) network. Quality of Service (QoS) requirements for SUs have significant implications on this protocol design. In this paper, we propose a new method to find joint policies for SUs which not only guarantees QoS requirements but also maximizes network throughput. We use Decentralized Partially Observable Markov Decision Process (Dec-POMDP) to formulate interactions between SUs. Meanwhile, a tractable approach for Dec-POMDP is utilized to extract sub-optimum joint policies for large horizons. Among these policies, the joint policy which guarantees QoS requirements is selected as the joint sensing strategy for SUs. To show the efficiency of the proposed method, we consider two SUs trying to access two-channel primary users (PUs) network modeled by discrete Markov chains. Simulations demonstrate three interesting findings: 1- Optimum joint policies for large horizons can be obtained using the proposed method. 2- There exists a joint policy for the assumed QoS constraints. 3- Our method outperforms other related works in terms of network throughput. △ Less

Submitted 14 October, 2010; originally announced October 2010.

A New Framework for Cognitive Medium Access Control: POSG Approach

Authors: Saber Salehkaleybar, Arash Majd, Mohammad Reza Pakravan

Abstract: In this paper, we propose a new analytical framework to solve medium access problem for secondary users (SUs) in cognitive radio networks. Partially Observable Stochastic Games (POSG) and Decentralized Markov Decision Process (Dec-POMDP) are two multi-agent Markovian decision processes which are used to present a solution. A primary network with two SUs is considered as an example to demonstrate o… ▽ More In this paper, we propose a new analytical framework to solve medium access problem for secondary users (SUs) in cognitive radio networks. Partially Observable Stochastic Games (POSG) and Decentralized Markov Decision Process (Dec-POMDP) are two multi-agent Markovian decision processes which are used to present a solution. A primary network with two SUs is considered as an example to demonstrate our proposed framework. Two different scenarios are assumed. In the first scenario, SUs compete to acquire the licensed channel which is modeled using POSG framework. In the second one, SUs cooperate to access channel for which the solution is based on Dec-POMDP. Besides, the dominant strategy for both of the above mentioned scenarios is presented for a three slot horizon length. △ Less

Submitted 14 October, 2010; v1 submitted 14 March, 2010; originally announced March 2010.

Comments: This paper has been withdrawn by the author due to minor contributions