-
An SMDP-Based Approach to Thermal-Aware Task Scheduling in NoC-based MPSoC platforms
Authors:
Farnaz Niknia,
Kiamehr Rezaee,
Vesal Hakami
Abstract:
One efficient approach to control chip-wide thermal distribution in multi-core systems is the optimization of online assignments of tasks to processing cores. Online task assignment, however, faces several uncertainties in real-world Systems and does not show a deterministic nature. In this paper, we consider the operation of a thermal-aware task scheduler, dispatching tasks from an arrival queue…
▽ More
One efficient approach to control chip-wide thermal distribution in multi-core systems is the optimization of online assignments of tasks to processing cores. Online task assignment, however, faces several uncertainties in real-world Systems and does not show a deterministic nature. In this paper, we consider the operation of a thermal-aware task scheduler, dispatching tasks from an arrival queue as well as setting the voltage and frequency of the processing cores to optimize the mean temperature margin of the entire chip (i.e., cores as well as the NoC routers). We model the decision process of the task scheduler as a semi-Markov decision problem (SMDP). Then, to solve the formulated SMDP, we propose two reinforcement learning algorithms that are capable of computing the optimal task assignment policy without requiring the statistical knowledge of the stochastic dynamics underlying the system states. The proposed algorithms also rely on function approximation techniques to handle the infinite length of the task queue as well as the continuous nature of temperature readings. Compared to related research, the simulation results show a nearly 6 Kelvin reduction in system average peak temperature and 66 milliseconds decrease in mean task service time.
△ Less
Submitted 6 September, 2020;
originally announced September 2020.
-
IEEE 802.15.4.e TSCH-Based Scheduling for Throughput Optimization: A Combinatorial Multi-Armed Bandit Approach
Authors:
Nastooh Taheri Javan,
Masoud Sabaei,
Vesal Hakami
Abstract:
In TSCH, which is a MAC mechanism set of the IEEE 802.15.4e amendment, calculation, construction, and maintenance of the packet transmission schedules are not defined. Moreover, to ensure optimal throughput, most of the existing scheduling methods are based on the assumption that instantaneous and accurate Channel State Information (CSI) is available. However, due to the inevitable errors in the c…
▽ More
In TSCH, which is a MAC mechanism set of the IEEE 802.15.4e amendment, calculation, construction, and maintenance of the packet transmission schedules are not defined. Moreover, to ensure optimal throughput, most of the existing scheduling methods are based on the assumption that instantaneous and accurate Channel State Information (CSI) is available. However, due to the inevitable errors in the channel estimation process, this assumption cannot be materialized in many practical scenarios. In this paper, we propose two alternative and realistic approaches. In our first approach, we assume that only the statistical knowledge of CSI is available a priori. Armed with this knowledge, the average packet rate on each link is computed and then, using the results, the throughput-optimal schedule for the assignment of (slot-frame) cells to links can be formulated as a max-weight bipartite matching problem, which can be solved efficiently using the well-known Hungarian algorithm. In the second approach, we assume that no CSI knowledge (even statistical) is available at the design stage. For this zero-knowledge setting, we introduce a machine learning-based algorithm by formally modeling the scheduling problem in terms of a combinatorial multi-armed bandit (CMAB) process. Our CMAB-based scheme is widely applicable to many real operational environments, thanks to its reduced reliance on design-time knowledge. Simulation results show that the average throughput obtained by the statistical CSI-based method is within the margin of 15% from the theoretical upper bound associated with perfect instantaneous CSI. The aforesaid margin is around 18% for our learning-theoretic solution.
△ Less
Submitted 14 September, 2019;
originally announced September 2019.
-
A Trust Management Scheme for IoT-Enabled Environmental Health/Accessibility Monitoring Services
Authors:
Behshid Shayesteh,
Vesal Hakami,
Ahmad Akbari
Abstract:
One rapidly growing application of Internet of Things (IoT) is the protection of public health and well-being through enabling environmental monitoring services. In particular, an IoT-enabled health/accessibility monitoring service (HAMS) can be consulted by its users to query about the status of different areas so as to optimize their trip throughout a geographic region. Given the high cost assoc…
▽ More
One rapidly growing application of Internet of Things (IoT) is the protection of public health and well-being through enabling environmental monitoring services. In particular, an IoT-enabled health/accessibility monitoring service (HAMS) can be consulted by its users to query about the status of different areas so as to optimize their trip throughout a geographic region. Given the high cost associated with a vast deployment of totally trusted information sources, the IoT-enabled monitoring services also subsist on citizen engagement and on (possibly untrusted) users' sensing apparatus for data collection. However, trust management becomes a key factor in the success of such services because they might be misled by malicious users through altered or fake sensor data. In this paper, we consider a monitoring service, and propose a hybrid entity/data trust computation scheme which relies on Bayesian learning to score the users (as data reporters), and on Dempster-Shafer theory (DST) for data fusion and for the computation of the trustworthiness of the data itself. In order to provide resiliency against behavioral changes, the probability masses used in DST are dynamically updated using the freshly estimated user scores as well as the contextual properties associated with the reported data. We conduct simulation experiments to evaluate the performance of our scheme. Compared to prior work, the results demonstrate superior performance in terms of accuracy and resilience against malicious behavior.
△ Less
Submitted 16 November, 2018;
originally announced November 2018.
-
A new rank-order clustering algorithm for prolonging the lifetime of wireless sensor networks
Authors:
Seyedakbar Mostafavi,
Vesal Hakami
Abstract:
Energy efficient resource management is critical for prolonging the lifetime of wireless sensor networks (WSN). Clustering of sensor nodes with the aim of distributing the traffic loads in the network is a proven approach for balanced energy consumption in WSN. The main body of literature in this topic can be classified as hierarchical and distance-based clustering techniques in which multi-hop, m…
▽ More
Energy efficient resource management is critical for prolonging the lifetime of wireless sensor networks (WSN). Clustering of sensor nodes with the aim of distributing the traffic loads in the network is a proven approach for balanced energy consumption in WSN. The main body of literature in this topic can be classified as hierarchical and distance-based clustering techniques in which multi-hop, multi-level forwarding and distance-based criteria, respectively, are utilized for categorization of sensor nodes. In this study, we propose the Approximate Rank-Order Wireless Sensor Networks (ARO-WSN) clustering algorithm as a combined hierarchical and distance-based clustering approach. ARO-WSN algorithm which has been extensively used in the field of image processing, runs in the order of O(n) for a large data set, therefore it can be applied on WSN. The results shows that ARO-WSN outperforms the classical LEACH, LEACH-C and K-means clustering algorithms in the terms of energy consumption and network lifetime.
△ Less
Submitted 4 October, 2019; v1 submitted 10 October, 2018;
originally announced October 2018.
-
A Stochastic Approximation Approach for Foresighted Task Scheduling in Cloud Computing
Authors:
Seyedakbar Mostafavi,
Vesal Hakami
Abstract:
With the increasing and elastic demand for cloud resources, finding an optimal task scheduling mechanism become a challenge for cloud service providers. Due to the time-varying nature of resource demands in length and processing over time and dynamics and heterogeneity of cloud resources, existing myopic task scheduling solutions intended to maximize the performance of task scheduling are ineffici…
▽ More
With the increasing and elastic demand for cloud resources, finding an optimal task scheduling mechanism become a challenge for cloud service providers. Due to the time-varying nature of resource demands in length and processing over time and dynamics and heterogeneity of cloud resources, existing myopic task scheduling solutions intended to maximize the performance of task scheduling are inefficient and sacrifice the long-time system performance in terms of resource utilization and response time. In this paper, we propose an optimal solution for performing foresighted task scheduling in a cloud environment. Since a-priori knowledge from the dynamics in queue length of virtual machines is not known in run time, an online reinforcement learning approach is proposed for foresighted task allocation. The evaluation results show that our method not only reduce the response time and makespan of submitted tasks, but also increase the resource efficiency. So in this thesis a scheduling method based on reinforcement learning is proposed. Adopting with environment conditions and responding to unsteady requests, reinforcement learning can cause a long-term increase in system's performance. The results show that this proposed method can not only reduce the response time and makespan but also increase resource efficiency as a minor goal.
△ Less
Submitted 11 April, 2020; v1 submitted 10 October, 2018;
originally announced October 2018.
-
Distributed Power Control for Delay Optimization in Energy Harvesting Cooperative Relay Networks
Authors:
Vesal Hakami,
Mehdi Dehghan
Abstract:
We consider cooperative communications with energy harvesting (EH) relays, and develop a distributed power control mechanism for the relaying terminals. Unlike prior art which mainly deal with single-relay systems with saturated traffic flow, we address the case of bursty data arrival at the source cooperatively forwarded by multiple half-duplex EH relays. We aim at optimizing the long-run average…
▽ More
We consider cooperative communications with energy harvesting (EH) relays, and develop a distributed power control mechanism for the relaying terminals. Unlike prior art which mainly deal with single-relay systems with saturated traffic flow, we address the case of bursty data arrival at the source cooperatively forwarded by multiple half-duplex EH relays. We aim at optimizing the long-run average delay of the source packets under the energy neutrality constraint on power consumption of each relay. While EH relay systems have been predominantly optimized using either offline or online methodologies, we take on a more realistic learning-theoretic approach. Hence, our scheme can be deployed for real-time operation without assuming acausal information on channel realizations, data/energy arrivals as required by offline optimization, nor does it rely on precise statistics of the system processes as is the case with online optimization. We formulate the problem as a partially observable identical payoff stochastic game (PO-IPSG) with factored controllers, in which the power control policy of each relay is adaptive to its local source-to-relay/relay-to-destination channel states, its local energy state as well as to the source buffer state information. We derive a multi-agent reinforcement learning algorithm which is convergent to a locally optimal solution of the formulated PO-IPSG. The proposed algorithm operates without explicit message exchange between the relays, while inducing only little source-relay signaling overhead. By simulation, we contrast the delay performance of the proposed method against existing heuristics for throughput maximization. It is shown that compared with these heuristics, the systematic approach adopted in this paper has a smaller sub-optimality gap once evaluated against a centralized optimal policy armed with perfect statistics.
△ Less
Submitted 24 October, 2018; v1 submitted 3 September, 2016;
originally announced September 2016.
-
Learning Stationary Correlated Equilibria in Constrained General-Sum Stochastic Games
Authors:
Vesal Hakami,
Mehdi Dehghan
Abstract:
We study constrained general-sum stochastic games with unknown Markovian dynamics. A distributed constrained no-regret Q-learning scheme (CNRQ) is presented to guarantee convergence to the set of stationary correlated equilibria of the game. Prior art addresses the unconstrained case only, is structured with nested control loops, and has no convergence result. CNRQ is cast as a single-loop three-t…
▽ More
We study constrained general-sum stochastic games with unknown Markovian dynamics. A distributed constrained no-regret Q-learning scheme (CNRQ) is presented to guarantee convergence to the set of stationary correlated equilibria of the game. Prior art addresses the unconstrained case only, is structured with nested control loops, and has no convergence result. CNRQ is cast as a single-loop three-timescale asynchronous stochastic approximation algorithm with set-valued update increments. A rigorous convergence analysis with differential inclusion arguments is given which draws on recent extensions of the theory of stochastic approximation to the case of asynchronous recursive inclusions with set-valued mean fields. Numerical results are given for the exemplary application of CNRQ to decentralized resource control in heterogeneous wireless networks (HetNets).
△ Less
Submitted 1 June, 2015; v1 submitted 5 February, 2014;
originally announced February 2014.
-
Cognitive Forwarding Control in Wireless Ad-Hoc Networks with Slow Fading Channels
Authors:
Vesal Hakami,
Mehdi Dehghan
Abstract:
We propose a decentralized stochastic control solution for the broadcast message dissemination problem in wireless ad hoc networks with slow fading channels. We formulate the control problem as a dynamic robust game which is well justified by two key observations; first, the shared nature of the wireless medium which inevitably cross-couples the nodes' forwarding decisions, thus binding them toget…
▽ More
We propose a decentralized stochastic control solution for the broadcast message dissemination problem in wireless ad hoc networks with slow fading channels. We formulate the control problem as a dynamic robust game which is well justified by two key observations; first, the shared nature of the wireless medium which inevitably cross-couples the nodes' forwarding decisions, thus binding them together as strategic players; second, the stochastic dynamics associated with the link qualities which renders the transmission costs noisy, thus motivating a robust formulation. Given the non stationarity induced by the fading process, an online solution for the formulated game would then require an adaptive procedure capable of both convergence to and tracking strategic equilibria as the environment changes. To this end, we deploy the strategic and non stationary learning algorithm of regret tracking, the temporally adaptive variant of the celebrated regret matching algorithm, to guarantee the emergence and active tracking of the correlated equilibria in the dynamic robust forwarding game. We also make provision for exploiting the channel state information, when available, to enhance the convergence speed of the learning algorithm by conducting an accurate transmission cost estimation. This cost estimate can basically serve as a model which spares the algorithm from extra action exploration, thus rendering the learning process more sample efficient. Simulation results reveal that our proposed solution excels in terms of both the number of transmissions and load distribution while also maintaining near perfect delivery ratio, especially in dense crowded environments.
△ Less
Submitted 1 June, 2015; v1 submitted 3 February, 2014;
originally announced February 2014.
