Showing 1–20 of 20 results for author: Calvo-Fullana, M

Multi-agent assignment via state augmented reinforcement learning

Authors: Leopoldo Agorio, Sean Van Alen, Miguel Calvo-Fullana, Santiago Paternain, Juan Andres Bazerque

Abstract: We address the conflicting requirements of a multi-agent assignment problem through constrained reinforcement learning, emphasizing the inadequacy of standard regularization techniques for this purpose. Instead, we recur to a state augmentation approach in which the oscillation of dual variables is exploited by agents to alternate between tasks. In addition, we coordinate the actions of the multip… ▽ More We address the conflicting requirements of a multi-agent assignment problem through constrained reinforcement learning, emphasizing the inadequacy of standard regularization techniques for this purpose. Instead, we recur to a state augmentation approach in which the oscillation of dual variables is exploited by agents to alternate between tasks. In addition, we coordinate the actions of the multiple agents acting on their local states through these multipliers, which are gossiped through a communication network, eliminating the need to access other agent states. By these means, we propose a distributed multi-agent assignment protocol with theoretical feasibility guarantees that we corroborate in a monitoring numerical experiment. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: 12 pages, 3 figures, 6th Annual Conference on Learning for Dynamics and Control

MSC Class: 93E35

Journal ref: Proceedings of Machine Learning Research vol 242 1 12, 2024. 6th Annual Conference on Learning for Dynamics and Control

A Networked Multi-Agent System for Mobile Wireless Infrastructure on Demand

Authors: Miguel Calvo-Fullana, Mikhail Gerasimenko, Daniel Mox, Leopoldo Agorio, Mariana del Castillo, Vijay Kumar, Alejandro Ribeiro, Juan Andres Bazerque

Abstract: Despite the prevalence of wireless connectivity in urban areas around the globe, there remain numerous and diverse situations where connectivity is insufficient or unavailable. To address this, we introduce mobile wireless infrastructure on demand, a system of UAVs that can be rapidly deployed to establish an ad-hoc wireless network. This network has the capability of reconfiguring itself dynamica… ▽ More Despite the prevalence of wireless connectivity in urban areas around the globe, there remain numerous and diverse situations where connectivity is insufficient or unavailable. To address this, we introduce mobile wireless infrastructure on demand, a system of UAVs that can be rapidly deployed to establish an ad-hoc wireless network. This network has the capability of reconfiguring itself dynamically to satisfy and maintain the required quality of communication. The system optimizes the positions of the UAVs and the routing of data flows throughout the network to achieve this quality of service (QoS). By these means, task agents using the network simply request a desired QoS, and the system adapts accordingly while allowing them to move freely. We have validated this system both in simulation and in real-world experiments. The results demonstrate that our system effectively offers mobile wireless infrastructure on demand, extending the operational range of task agents and supporting complex mobility patterns, all while ensuring connectivity and being resilient to agent failures. △ Less

Submitted 14 June, 2023; originally announced June 2023.

Mission-Aware Value of Information Censoring for Distributed Filtering

Authors: Miguel Calvo-Fullana, Jonathan P. How

Abstract: In this paper, we study the problem of distributed estimation with an emphasis on communication-efficiency. The proposed algorithm is based on a windowed maximum a posteriori (MAP) estimation problem, wherein each agent in the network locally computes a Kalman-like filter estimate that approximates the centralized MAP solution. Information sharing among agents is restricted to their neighbors only… ▽ More In this paper, we study the problem of distributed estimation with an emphasis on communication-efficiency. The proposed algorithm is based on a windowed maximum a posteriori (MAP) estimation problem, wherein each agent in the network locally computes a Kalman-like filter estimate that approximates the centralized MAP solution. Information sharing among agents is restricted to their neighbors only, with guarantees on overall estimate consistency provided via logarithmic opinion pooling. The problem is efficiently distributed using the alternating direction method of multipliers (ADMM), whose overall communication usage is further reduced by a value of information (VoI) censoring mechanism, wherein agents only transmit their primal-dual iterates when deemed valuable to do so. The proposed censoring mechanism is mission-aware, enabling a globally efficient use of communication resources while guaranteeing possibly different local estimation requirements. To illustrate the validity of the approach we perform simulations in a target tracking scenario. △ Less

Submitted 20 November, 2022; originally announced November 2022.

Multi-task Bias-Variance Trade-off Through Functional Constraints

Authors: Juan Cervino, Juan Andres Bazerque, Miguel Calvo-Fullana, Alejandro Ribeiro

Abstract: Multi-task learning aims to acquire a set of functions, either regressors or classifiers, that perform well for diverse tasks. At its core, the idea behind multi-task learning is to exploit the intrinsic similarity across data sources to aid in the learning process for each individual domain. In this paper we draw intuition from the two extreme learning scenarios -- a single function for all tasks… ▽ More Multi-task learning aims to acquire a set of functions, either regressors or classifiers, that perform well for diverse tasks. At its core, the idea behind multi-task learning is to exploit the intrinsic similarity across data sources to aid in the learning process for each individual domain. In this paper we draw intuition from the two extreme learning scenarios -- a single function for all tasks, and a task-specific function that ignores the other tasks dependencies -- to propose a bias-variance trade-off. To control the relationship between the variance (given by the number of i.i.d. samples), and the bias (coming from data from other task), we introduce a constrained learning formulation that enforces domain specific solutions to be close to a central function. This problem is solved in the dual domain, for which we propose a stochastic primal-dual algorithm. Experimental results for a multi-domain classification problem with real data show that the proposed procedure outperforms both the task specific, as well as the single classifiers. △ Less

Submitted 27 October, 2022; originally announced October 2022.

Distributed Filtering with Value of Information Censoring

Authors: Miguel Calvo-Fullana, Jonathan P. How

Abstract: This work presents a distributed estimation algorithm that efficiently uses the available communication resources. The approach is based on Bayesian filtering that is distributed across a network by using the logarithmic opinion pool operator. Communication efficiency is achieved by having only agents with high Value of Information (VoI) share their estimates, and the algorithm provides a tunable… ▽ More This work presents a distributed estimation algorithm that efficiently uses the available communication resources. The approach is based on Bayesian filtering that is distributed across a network by using the logarithmic opinion pool operator. Communication efficiency is achieved by having only agents with high Value of Information (VoI) share their estimates, and the algorithm provides a tunable trade-off between communication resources and estimation error. Under linear-Gaussian models the algorithm takes the form of a censored distributed Information filter, which guarantees the consistency of agent estimates. Importantly, consistent estimates are shown to play a crucial role in enabling the large reductions in communication usage provided by the VoI censoring approach. We verify the performance of the proposed method via complex simulations in a dynamic network topology and by experimental validation over a real ad-hoc wireless communication network. The results show the validity of using the proposed method to drastically reduce the communication costs of distributed estimation tasks. △ Less

Submitted 1 April, 2022; originally announced April 2022.

Distributed Riemannian Optimization with Lazy Communication for Collaborative Geometric Estimation

Authors: Yulun Tian, Amrit Singh Bedi, Alec Koppel, Miguel Calvo-Fullana, David M. Rosen, Jonathan P. How

Abstract: We present the first distributed optimization algorithm with lazy communication for collaborative geometric estimation, the backbone of modern collaborative simultaneous localization and map** (SLAM) and structure-from-motion (SfM) applications. Our method allows agents to cooperatively reconstruct a shared geometric model on a central server by fusing individual observations, but without the ne… ▽ More We present the first distributed optimization algorithm with lazy communication for collaborative geometric estimation, the backbone of modern collaborative simultaneous localization and map** (SLAM) and structure-from-motion (SfM) applications. Our method allows agents to cooperatively reconstruct a shared geometric model on a central server by fusing individual observations, but without the need to transmit potentially sensitive information about the agents themselves (such as their locations). Furthermore, to alleviate the burden of communication during iterative optimization, we design a set of communication triggering conditions that enable agents to selectively upload a targeted subset of local information that is useful to global optimization. Our approach thus achieves significant communication reduction with minimal impact on optimization performance. As our main theoretical contribution, we prove that our method converges to first-order critical points with a global sublinear convergence rate. Numerical evaluations on bundle adjustment problems from collaborative SLAM and SfM datasets show that our method performs competitively against existing distributed techniques, while achieving up to 78% total communication reduction. △ Less

Submitted 29 July, 2022; v1 submitted 1 March, 2022; originally announced March 2022.

Comments: technical report (17 pages, 3 figures); to appear at IROS 2022

Constrained Learning with Non-Convex Losses

Authors: Luiz F. O. Chamon, Santiago Paternain, Miguel Calvo-Fullana, Alejandro Ribeiro

Abstract: Though learning has become a core component of modern information processing, there is now ample evidence that it can lead to biased, unsafe, and prejudiced systems. The need to impose requirements on learning is therefore paramount, especially as it reaches critical applications in social, industrial, and medical domains. However, the non-convexity of most modern statistical problems is only exac… ▽ More Though learning has become a core component of modern information processing, there is now ample evidence that it can lead to biased, unsafe, and prejudiced systems. The need to impose requirements on learning is therefore paramount, especially as it reaches critical applications in social, industrial, and medical domains. However, the non-convexity of most modern statistical problems is only exacerbated by the introduction of constraints. Whereas good unconstrained solutions can often be learned using empirical risk minimization, even obtaining a model that satisfies statistical constraints can be challenging. All the more so, a good one. In this paper, we overcome this issue by learning in the empirical dual domain, where constrained statistical learning problems become unconstrained and deterministic. We analyze the generalization properties of this approach by bounding the empirical duality gap -- i.e., the difference between our approximate, tractable solution and the solution of the original (non-convex) statistical problem -- and provide a practical constrained learning algorithm. These results establish a constrained counterpart to classical learning theory, enabling the explicit use of constraints in learning. We illustrate this theory and algorithm in rate-constrained learning applications arising in fairness and adversarial robustness. △ Less

Submitted 19 October, 2022; v1 submitted 8 March, 2021; originally announced March 2021.

Comments: IEEE Transactions on Information Theory

Towards Safe Continuing Task Reinforcement Learning

Authors: Miguel Calvo-Fullana, Luiz F. O. Chamon, Santiago Paternain

Abstract: Safety is a critical feature of controller design for physical systems. When designing control policies, several approaches to guarantee this aspect of autonomy have been proposed, such as robust controllers or control barrier functions. However, these solutions strongly rely on the model of the system being available to the designer. As a parallel development, reinforcement learning provides mode… ▽ More Safety is a critical feature of controller design for physical systems. When designing control policies, several approaches to guarantee this aspect of autonomy have been proposed, such as robust controllers or control barrier functions. However, these solutions strongly rely on the model of the system being available to the designer. As a parallel development, reinforcement learning provides model-agnostic control solutions but in general, it lacks the theoretical guarantees required for safety. Recent advances show that under mild conditions, control policies can be learned via reinforcement learning, which can be guaranteed to be safe by imposing these requirements as constraints of an optimization problem. However, to transfer from learning safety to learning safely, there are two hurdles that need to be overcome: (i) it has to be possible to learn the policy without having to re-initialize the system; and (ii) the rollouts of the system need to be in themselves safe. In this paper, we tackle the first issue, proposing an algorithm capable of operating in the continuing task setting without the need of restarts. We evaluate our approach in a numerical example, which shows the capabilities of the proposed approach in learning safe policies via safe exploration. △ Less

Submitted 24 February, 2021; originally announced February 2021.

State Augmented Constrained Reinforcement Learning: Overcoming the Limitations of Learning with Rewards

Authors: Miguel Calvo-Fullana, Santiago Paternain, Luiz F. O. Chamon, Alejandro Ribeiro

Abstract: A common formulation of constrained reinforcement learning involves multiple rewards that must individually accumulate to given thresholds. In this class of problems, we show a simple example in which the desired optimal policy cannot be induced by any weighted linear combination of rewards. Hence, there exist constrained reinforcement learning problems for which neither regularized nor classical… ▽ More A common formulation of constrained reinforcement learning involves multiple rewards that must individually accumulate to given thresholds. In this class of problems, we show a simple example in which the desired optimal policy cannot be induced by any weighted linear combination of rewards. Hence, there exist constrained reinforcement learning problems for which neither regularized nor classical primal-dual methods yield optimal policies. This work addresses this shortcoming by augmenting the state with Lagrange multipliers and reinterpreting primal-dual methods as the portion of the dynamics that drives the multipliers evolution. This approach provides a systematic state augmentation procedure that is guaranteed to solve reinforcement learning problems with constraints. Thus, as we illustrate by an example, while previous methods can fail at finding optimal policies, running the dual dynamics while executing the augmented policy yields an algorithm that provably samples actions from the optimal policy. △ Less

Submitted 21 September, 2023; v1 submitted 23 February, 2021; originally announced February 2021.

ROS-NetSim: A Framework for the Integration of Robotic and Network Simulators

Authors: Miguel Calvo-Fullana, Daniel Mox, Alexander Pyattaev, Jonathan Fink, Vijay Kumar, Alejandro Ribeiro

Abstract: Multi-agent systems play an important role in modern robotics. Due to the nature of these systems, coordination among agents via communication is frequently necessary. Indeed, Perception-Action-Communication (PAC) loops, or Perception-Action loops closed over a communication channel, are a critical component of multi-robot systems. However, we lack appropriate tools for simulating PAC loops. To th… ▽ More Multi-agent systems play an important role in modern robotics. Due to the nature of these systems, coordination among agents via communication is frequently necessary. Indeed, Perception-Action-Communication (PAC) loops, or Perception-Action loops closed over a communication channel, are a critical component of multi-robot systems. However, we lack appropriate tools for simulating PAC loops. To that end, in this paper, we introduce ROS-NetSim, a ROS package that acts as an interface between robotic and network simulators. With ROS-NetSim, we can attain high-fidelity representations of both robotic and network interactions by accurately simulating the PAC loop. Our proposed approach is lightweight, modular and adaptive. Furthermore, it can be used with many available network and physics simulators by making use of our proposed interface. In summary, ROS-NetSim is (i) Transparent to the ROS target application, (ii) Agnostic to the specific network and physics simulator being used, and (iii) Tunable in fidelity and complexity. As part of our contribution, we have made available an open-source implementation of ROS-NetSim to the community. △ Less

Submitted 25 January, 2021; originally announced January 2021.

Multi-task Supervised Learning via Cross-learning

Authors: Juan Cervino, Juan Andres Bazerque, Miguel Calvo-Fullana, Alejandro Ribeiro

Abstract: In this paper we consider a problem known as multi-task learning, consisting of fitting a set of classifier or regression functions intended for solving different tasks. In our novel formulation, we couple the parameters of these functions, so that they learn in their task specific domains while staying close to each other. This facilitates cross-fertilization in which data collected across differ… ▽ More In this paper we consider a problem known as multi-task learning, consisting of fitting a set of classifier or regression functions intended for solving different tasks. In our novel formulation, we couple the parameters of these functions, so that they learn in their task specific domains while staying close to each other. This facilitates cross-fertilization in which data collected across different domains help improving the learning performance at each other task. First, we present a simplified case in which the goal is to estimate the means of two Gaussian variables, for the purpose of gaining some insights on the advantage of the proposed cross-learning strategy. Then we provide a stochastic projected gradient algorithm to perform cross-learning over a generic loss function. If the number of parameters is large, then the projection step becomes computationally expensive. To avoid this situation, we derive a primal-dual algorithm that exploits the structure of the dual problem, achieving a formulation whose complexity only depends on the number of tasks. Preliminary numerical experiments for image classification by neural networks trained on a dataset divided in different domains corroborate that the cross-learned function outperforms both the task-specific and the consensus approaches. △ Less

Submitted 26 May, 2021; v1 submitted 24 October, 2020; originally announced October 2020.

Multi-task Reinforcement Learning in Reproducing Kernel Hilbert Spaces via Cross-learning

Authors: Juan Cervino, Juan Andres Bazerque, Miguel Calvo-Fullana, Alejandro Ribeiro

Abstract: Reinforcement learning (RL) is a framework to optimize a control policy using rewards that are revealed by the system as a response to a control action. In its standard form, RL involves a single agent that uses its policy to accomplish a specific task. These methods require large amounts of reward samples to achieve good performance, and may not generalize well when the task is modified, even if… ▽ More Reinforcement learning (RL) is a framework to optimize a control policy using rewards that are revealed by the system as a response to a control action. In its standard form, RL involves a single agent that uses its policy to accomplish a specific task. These methods require large amounts of reward samples to achieve good performance, and may not generalize well when the task is modified, even if the new task is related. In this paper we are interested in a collaborative scheme in which multiple agents with different tasks optimize their policies jointly. To this end, we introduce cross-learning, in which agents tackling related tasks have their policies constrained to be close to one another. Two properties make our new approach attractive: (i) it produces a multi-task central policy that can be used as a starting point to adapt quickly to one of the tasks trained for, in a situation when the agent does not know which task is currently facing, and (ii) as in meta-learning, it adapts to environments related but different to those seen during training. We focus on continuous policies belonging to reproducing kernel Hilbert spaces for which we bound the distance between the task-specific policies and the cross-learned policy. To solve the resulting optimization problem, we resort to a projected policy gradient algorithm and prove that it converges to a near-optimal solution with high probability. We evaluate our methodology with a navigation example in which agents can move through environments with obstacles of multiple shapes and avoid obstacles not trained for. △ Less

Submitted 26 August, 2020; originally announced August 2020.

The empirical duality gap of constrained statistical learning

Authors: Luiz F. O. Chamon, Santiago Paternain, Miguel Calvo-Fullana, Alejandro Ribeiro

Abstract: This paper is concerned with the study of constrained statistical learning problems, the unconstrained version of which are at the core of virtually all of modern information processing. Accounting for constraints, however, is paramount to incorporate prior knowledge and impose desired structural and statistical properties on the solutions. Still, solving constrained statistical problems remains c… ▽ More This paper is concerned with the study of constrained statistical learning problems, the unconstrained version of which are at the core of virtually all of modern information processing. Accounting for constraints, however, is paramount to incorporate prior knowledge and impose desired structural and statistical properties on the solutions. Still, solving constrained statistical problems remains challenging and guarantees scarce, leaving them to be tackled using regularized formulations. Though practical and effective, selecting regularization parameters so as to satisfy requirements is challenging, if at all possible, due to the lack of a straightforward relation between parameters and constraints. In this work, we propose to directly tackle the constrained statistical problem overcoming its infinite dimensionality, unknown distributions, and constraints by leveraging finite dimensional parameterizations, sample averages, and duality theory. Aside from making the problem tractable, these tools allow us to bound the empirical duality gap, i.e., the difference between our approximate tractable solutions and the actual solutions of the original statistical problem. We demonstrate the effectiveness and usefulness of this constrained formulation in a fair learning application. △ Less

Submitted 12 February, 2020; originally announced February 2020.

Mobile Wireless Network Infrastructure on Demand

Authors: Daniel Mox, Miguel Calvo-Fullana, Mikhail Gerasimenko, Jonathan Fink, Vijay Kumar, Alejandro Ribeiro

Abstract: In this work, we introduce Mobile Wireless In-frastructure on Demand: a framework for providing wireless connectivity to multi-robot teams via autonomously reconfiguring ad-hoc networks. In many cases, previous multi-agent systems either assumed the availability of existing communication infrastructure or were required to create a network in addition to completing their objective. Instead our syst… ▽ More In this work, we introduce Mobile Wireless In-frastructure on Demand: a framework for providing wireless connectivity to multi-robot teams via autonomously reconfiguring ad-hoc networks. In many cases, previous multi-agent systems either assumed the availability of existing communication infrastructure or were required to create a network in addition to completing their objective. Instead our system explicitly assumes the responsibility of creating and sustaining a wireless network capable of satisfying end-to-end communication requirements of a team of agents, called the task team, performing an arbitrary objective. To accomplish this goal, we propose a joint optimization framework that alternates between finding optimal network routes to support data flows between the task agents and improving the performance of the network by repositioning a collection of mobile relay nodes referred to as the network team. We demonstrate our approach with simulations and experiments wherein wireless connectivity is provided to patrolling task agents. △ Less

Submitted 24 March, 2020; v1 submitted 7 February, 2020; originally announced February 2020.

Comments: 7 pages, 7 figures, accepted to ICRA 2020

Safe Policies for Reinforcement Learning via Primal-Dual Methods

Authors: Santiago Paternain, Miguel Calvo-Fullana, Luiz F. O. Chamon, Alejandro Ribeiro

Abstract: In this paper, we study the learning of safe policies in the setting of reinforcement learning problems. This is, we aim to control a Markov Decision Process (MDP) of which we do not know the transition probabilities, but we have access to sample trajectories through experience. We define safety as the agent remaining in a desired safe set with high probability during the operation time. We theref… ▽ More In this paper, we study the learning of safe policies in the setting of reinforcement learning problems. This is, we aim to control a Markov Decision Process (MDP) of which we do not know the transition probabilities, but we have access to sample trajectories through experience. We define safety as the agent remaining in a desired safe set with high probability during the operation time. We therefore consider a constrained MDP where the constraints are probabilistic. Since there is no straightforward way to optimize the policy with respect to the probabilistic constraint in a reinforcement learning framework, we propose an ergodic relaxation of the problem. The advantages of the proposed relaxation are threefold. (i) The safety guarantees are maintained in the case of episodic tasks and they are kept up to a given time horizon for continuing tasks. (ii) The constrained optimization problem despite its non-convexity has arbitrarily small duality gap if the parametrization of the policy is rich enough. (iii) The gradients of the Lagrangian associated with the safe-learning problem can be easily computed using standard policy gradient results and stochastic approximation tools. Leveraging these advantages, we establish that primal-dual algorithms are able to find policies that are safe and optimal. We test the proposed approach in a navigation task in a continuous domain. The numerical results show that our algorithm is capable of dynamically adapting the policy to the environment and the required safety levels. △ Less

Submitted 12 January, 2022; v1 submitted 20 November, 2019; originally announced November 2019.

Comments: arXiv admin note: text overlap with arXiv:1910.13393

Constrained Reinforcement Learning Has Zero Duality Gap

Authors: Santiago Paternain, Luiz F. O. Chamon, Miguel Calvo-Fullana, Alejandro Ribeiro

Abstract: Autonomous agents must often deal with conflicting requirements, such as completing tasks using the least amount of time/energy, learning multiple tasks, or dealing with multiple opponents. In the context of reinforcement learning~(RL), these problems are addressed by (i)~designing a reward function that simultaneously describes all requirements or (ii)~combining modular value functions that encod… ▽ More Autonomous agents must often deal with conflicting requirements, such as completing tasks using the least amount of time/energy, learning multiple tasks, or dealing with multiple opponents. In the context of reinforcement learning~(RL), these problems are addressed by (i)~designing a reward function that simultaneously describes all requirements or (ii)~combining modular value functions that encode them individually. Though effective, these methods have critical downsides. Designing good reward functions that balance different objectives is challenging, especially as the number of objectives grows. Moreover, implicit interference between goals may lead to performance plateaus as they compete for resources, particularly when training on-policy. Similarly, selecting parameters to combine value functions is at least as hard as designing an all-encompassing reward, given that the effect of their values on the overall policy is not straightforward. The later is generally addressed by formulating the conflicting requirements as a constrained RL problem and solved using Primal-Dual methods. These algorithms are in general not guaranteed to converge to the optimal solution since the problem is not convex. This work provides theoretical support to these approaches by establishing that despite its non-convexity, this problem has zero duality gap, i.e., it can be solved exactly in the dual domain, where it becomes convex. Finally, we show this result basically holds if the policy is described by a good parametrization~(e.g., neural networks) and we connect this result with primal-dual algorithms present in the literature and we establish the convergence to the optimal solution. △ Less

Submitted 29 October, 2019; originally announced October 2019.

Random Access Communication for Wireless Control Systems with Energy Harvesting Sensors

Authors: Miguel Calvo-Fullana, Carles Antón-Haro, Javier Matamoros, Alejandro Ribeiro

Abstract: In this paper, we study wireless networked control systems in which the sensing devices are powered by energy harvesting. We consider a scenario with multiple plants, where the sensors communicate their measurements to their respective controllers over a shared wireless channel. Due to the shared nature of the medium, sensors transmitting simultaneously can lead to packet collisions. In order to d… ▽ More In this paper, we study wireless networked control systems in which the sensing devices are powered by energy harvesting. We consider a scenario with multiple plants, where the sensors communicate their measurements to their respective controllers over a shared wireless channel. Due to the shared nature of the medium, sensors transmitting simultaneously can lead to packet collisions. In order to deal with this, we propose the use of random access communication policies and, to this end, we translate the control performance requirements to successful packet reception probabilities. The optimal scheduling decision is to transmit with a certain probability, which is adaptive to plant, channel and battery conditions. Moreover, we provide a stochastic dual method to compute the optimal scheduling solution, which is decoupled across sensors, with only some of the dual variables needed to be shared between nodes. Furthermore, we also consider asynchronicity in the values of the variables across sensor nodes and provide theoretical guarantees on the stability of the control systems under the proposed random access mechanism. Finally, we provide extensive numerical results that corroborate our claims. △ Less

Submitted 30 January, 2018; originally announced January 2018.

Stochastic Routing and Scheduling Policies for Energy Harvesting Communication Networks

Authors: Miguel Calvo-Fullana, Carles Antón-Haro, Javier Matamoros, Alejandro Ribeiro

Abstract: In this paper, we study the joint routing-scheduling problem in energy harvesting communication networks. Our policies, which are based on stochastic subgradient methods on the dual domain, act as an energy harvesting variant of the stochastic family of backpresure algorithms. Specifically, we propose two policies: (i) the Stochastic Backpressure with Energy Harvesting (SBP-EH), in which a node's… ▽ More In this paper, we study the joint routing-scheduling problem in energy harvesting communication networks. Our policies, which are based on stochastic subgradient methods on the dual domain, act as an energy harvesting variant of the stochastic family of backpresure algorithms. Specifically, we propose two policies: (i) the Stochastic Backpressure with Energy Harvesting (SBP-EH), in which a node's routing-scheduling decisions are determined by the difference between the Lagrange multipliers associated to their queue stability constraints and their neighbors'; and (ii) the Stochastic Soft Backpressure with Energy Harvesting (SSBP-EH), an improved algorithm where the routing-scheduling decision is of a probabilistic nature. For both policies, we show that given sustainable data and energy arrival rates, the stability of the data queues over all network nodes is guaranteed. Numerical results corroborate the stability guarantees and illustrate the minimal gap in performance that our policies offer with respect to classical ones which work with an unlimited energy supply. △ Less

Submitted 2 November, 2017; originally announced November 2017.

Reconstruction of Correlated Sources with Energy Harvesting Constraints in Delay-constrained and Delay-tolerant Communication Scenarios

Authors: Miguel Calvo-Fullana, Javier Matamoros, Carles Antón-Haro

Abstract: In this paper, we investigate the reconstruction of time-correlated sources in a point-to-point communications scenario comprising an energy-harvesting sensor and a Fusion Center (FC). Our goal is to minimize the average distortion in the reconstructed observations by using data from previously encoded sources as side information. First, we analyze a delay-constrained scenario, where the sources m… ▽ More In this paper, we investigate the reconstruction of time-correlated sources in a point-to-point communications scenario comprising an energy-harvesting sensor and a Fusion Center (FC). Our goal is to minimize the average distortion in the reconstructed observations by using data from previously encoded sources as side information. First, we analyze a delay-constrained scenario, where the sources must be reconstructed before the next time slot. We formulate the problem in a convex optimization framework and derive the optimal transmission (i.e., power and rate allocation) policy. To solve this problem, we propose an iterative algorithm based on the subgradient method. Interestingly, the solution to the problem consists of a coupling between a two-dimensional directional water-filling algorithm (for power allocation) and a reverse water-filling algorithm (for rate allocation). Then we find a more general solution to this problem in a delay-tolerant scenario where the time horizon for source reconstruction is extended to multiple time slots. Finally, we provide some numerical results that illustrate the impact of delay and correlation in the power and rate allocation policies, and in the resulting reconstruction distortion. We also discuss the performance gap exhibited by a heuristic online policy derived from the optimal (offline) one. △ Less

Submitted 24 January, 2017; originally announced January 2017.

Sensor Selection and Power Allocation Strategies for Energy Harvesting Wireless Sensor Networks

Authors: Miguel Calvo-Fullana, Javier Matamoros, Carles Antón-Haro

Abstract: In this paper, we investigate the problem of jointly selecting a predefined number of energy-harvesting (EH) sensors and computing the optimal power allocation. The ultimate goal is to minimize the reconstruction distortion at the fusion center. This optimization problem is, unfortunately, non-convex. To circumvent that, we propose two suboptimal strategies: (i) a joint sensor selection and power… ▽ More In this paper, we investigate the problem of jointly selecting a predefined number of energy-harvesting (EH) sensors and computing the optimal power allocation. The ultimate goal is to minimize the reconstruction distortion at the fusion center. This optimization problem is, unfortunately, non-convex. To circumvent that, we propose two suboptimal strategies: (i) a joint sensor selection and power allocation (JSS-EH) scheme that, we prove, is capable of iteratively finding a stationary solution of the original problem from a sequence of surrogate convex problems; and (ii) a separate sensor selection and power allocation (SS-EH) scheme, on which basis we can identify a sensible sensor selection and analytically find a power allocation policy by solving a convex problem. We also discuss the interplay between the two strategies. Performance in terms of reconstruction distortion, impact of initialization, actual subsets of selected sensors and computed power allocation policies, etc., is assessed by means of computer simulations. To that aim, an EH-agnostic sensor selection strategy, a lower bound on distortion, and an online version of the SS-EH and JSS-EH schemes are derived and used for benchmarking. △ Less

Submitted 12 August, 2016; originally announced August 2016.