-
Reinforcement Learning-driven Data-intensive Workflow Scheduling for Volunteer Edge-Cloud
Authors:
Motahare Mounesan,
Mauro Lemus,
Hemanth Yeddulapalli,
Prasad Calyam,
Saptarshi Debroy
Abstract:
In recent times, Volunteer Edge-Cloud (VEC) has gained traction as a cost-effective, community computing paradigm to support data-intensive scientific workflows. However, due to the highly distributed and heterogeneous nature of VEC resources, centralized workflow task scheduling remains a challenge. In this paper, we propose a Reinforcement Learning (RL)-driven data-intensive scientific workflow…
▽ More
In recent times, Volunteer Edge-Cloud (VEC) has gained traction as a cost-effective, community computing paradigm to support data-intensive scientific workflows. However, due to the highly distributed and heterogeneous nature of VEC resources, centralized workflow task scheduling remains a challenge. In this paper, we propose a Reinforcement Learning (RL)-driven data-intensive scientific workflow scheduling approach that takes into consideration: i) workflow requirements, ii) VEC resources' preference on workflows, and iii) diverse VEC resource policies, to ensure robust resource allocation. We formulate the long-term average performance optimization problem as a Markov Decision Process, which is solved using an event-based Asynchronous Advantage Actor-Critic RL approach. Our extensive simulations and testbed implementations demonstrate our approach's benefits over popular baseline strategies in terms of workflow requirement satisfaction, VEC preference satisfaction, and available VEC resource utilization.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
End-to-End Latency Optimization of Multi-view 3D Reconstruction for Disaster Response
Authors:
Xiaojie Zhang,
Mingjun Li,
Andrew Hilton,
Amitangshu Pal,
Soumyabrata Dey,
Saptarshi Debroy
Abstract:
In order to plan rapid response during disasters, first responder agencies often adopt `bring your own device' (BYOD) model with inexpensive mobile edge devices (e.g., drones, robots, tablets) for complex video analytics applications, e.g., 3D reconstruction of a disaster scene. Unlike simpler video applications, widely used Multi-view Stereo (MVS) based 3D reconstruction applications (e.g., openM…
▽ More
In order to plan rapid response during disasters, first responder agencies often adopt `bring your own device' (BYOD) model with inexpensive mobile edge devices (e.g., drones, robots, tablets) for complex video analytics applications, e.g., 3D reconstruction of a disaster scene. Unlike simpler video applications, widely used Multi-view Stereo (MVS) based 3D reconstruction applications (e.g., openMVG/openMVS) are exceedingly time consuming, especially when run on such computationally constrained mobile edge devices. Additionally, reducing the reconstruction latency of such inherently sequential algorithms is challenging as unintelligent, application-agnostic strategies can drastically degrade the reconstruction (i.e., application outcome) quality making them useless. In this paper, we aim to design a latency optimized MVS algorithm pipeline, with the objective to best balance the end-to-end latency and reconstruction quality by running the pipeline on a collaborative mobile edge environment. The overall optimization approach is two-pronged where: (a) application optimizations introduce data-level parallelism by splitting the pipeline into high frequency and low frequency reconstruction components and (b) system optimizations incorporate task-level parallelism to the pipelines by running them opportunistically on available resources with online quality control in order to balance both latency and quality. Our evaluation on a hardware testbed using publicly available datasets shows upto ~54% reduction in latency with negligible loss (~4-7%) in reconstruction quality.
△ Less
Submitted 3 April, 2023;
originally announced April 2023.
-
DNN-based Denial of Quality of Service Attack on Software-defined Hybrid Edge-Cloud Systems
Authors:
Minh Nguyen,
Jacob Gately,
Swati Kar,
Soumyabrata Dey,
Saptarshi Debroy
Abstract:
In order to satisfy diverse quality-of-service (QoS) requirements of complex real-time video applications, civilian and tactical use cases are employing software-defined hybrid edge-cloud systems. One of the primary QoS requirements of such applications is ultra-low end-to-end latency for video applications that necessitates rapid frame transfer between end-devices and edge servers using software-…
▽ More
In order to satisfy diverse quality-of-service (QoS) requirements of complex real-time video applications, civilian and tactical use cases are employing software-defined hybrid edge-cloud systems. One of the primary QoS requirements of such applications is ultra-low end-to-end latency for video applications that necessitates rapid frame transfer between end-devices and edge servers using software-defined networking (SDN). Failing to guarantee such strict requirements leads to quality degradation of video applications and subsequently mission failure. In this paper, we show how a collaborative group of attackers can exploit SDN's control communications to launch Denial of Quality of Service (DQoS) attack that artificially increases end-to-end latency of video frames and yet evades detection. In particular, we show how Deep Neural Network (DNN) model training on all or partial network state information can help predict network packet drop rates with reasonable accuracy. We also show how such predictions can help design an attack model that can inflict just the right amount of added latency to the end-to-end video processing that is enough to cause considerable QoS degradation but not too much to raise suspicion. We use a realistic edge-cloud testbed on GENI platform for training data collection and demonstration of high model accuracy and attack success rate.
△ Less
Submitted 2 April, 2023;
originally announced April 2023.
-
Challenges in Modeling Complexity of Neglected Tropical Diseases: Assessment of Visceral Leishmaniasis Dynamics in Resource Limited Settings
Authors:
Swati DebRoy,
Olivia Prosper,
Austin Mishoe,
Anuj Mubayi
Abstract:
Neglected tropical diseases (NTD), particularly vector-borne diseases (VBD), account for a large proportion of the global disease burden, and their control faces several challenges including diminishing human and financial resources for those distressed from such diseases. Visceral Leishmaniasis (VL), the second-largest parasitic killer in the world (after malaria) affects poor populations in ende…
▽ More
Neglected tropical diseases (NTD), particularly vector-borne diseases (VBD), account for a large proportion of the global disease burden, and their control faces several challenges including diminishing human and financial resources for those distressed from such diseases. Visceral Leishmaniasis (VL), the second-largest parasitic killer in the world (after malaria) affects poor populations in endemic countries and causes considerable cost to the affected individuals and their society. Mathematical models can serve as a critical tool for understanding the driving mechanisms of a NTD such as VL. The WHO promotes integrated control programs for VL but this policy is not well supported by systematic quantitative and dynamic evidence and so potential benefits of the policy are limited. Moreover, mathematical models can be readily developed and used to understand the functioning of the VL system cheaply and systematically. The focus of this research is three-fold: (i) to identify non-traditional but critical mechanisms for ongoing VL transmission in resource limited regions, (ii) to review mathematical models used for other infectious diseases that have the potential to capture identified factors of VL, and (iii) to suggest novel quantitative models for understanding VL dynamics and for evaluating control programs in such frameworks for achieveing VL elimination goals.
△ Less
Submitted 2 September, 2016;
originally announced September 2016.
