-
CAMEO: A Causal Transfer Learning Approach for Performance Optimization of Configurable Computer Systems
Authors:
Md Shahriar Iqbal,
Ziyuan Zhong,
Iftakhar Ahmad,
Baishakhi Ray,
Pooyan Jamshidi
Abstract:
Modern computer systems are highly configurable, with hundreds of configuration options that interact, resulting in an enormous configuration space. As a result, optimizing performance goals (e.g., latency) in such systems is challenging due to frequent uncertainties in their environments (e.g., workload fluctuations). Recently, transfer learning has been applied to address this problem by reusing…
▽ More
Modern computer systems are highly configurable, with hundreds of configuration options that interact, resulting in an enormous configuration space. As a result, optimizing performance goals (e.g., latency) in such systems is challenging due to frequent uncertainties in their environments (e.g., workload fluctuations). Recently, transfer learning has been applied to address this problem by reusing knowledge from configuration measurements from the source environments, where it is cheaper to intervene than the target environment, where any intervention is costly or impossible. Recent empirical research showed that statistical models can perform poorly when the deployment environment changes because the behavior of certain variables in the models can change dramatically from source to target. To address this issue, we propose CAMEO, a method that identifies invariant causal predictors under environmental changes, allowing the optimization process to operate in a reduced search space, leading to faster optimization of system performance. We demonstrate significant performance improvements over state-of-the-art optimization methods in MLperf deep learning systems, a video analytics pipeline, and a database system.
△ Less
Submitted 3 October, 2023; v1 submitted 13 June, 2023;
originally announced June 2023.
-
Unicorn: Reasoning about Configurable System Performance through the lens of Causality
Authors:
Md Shahriar Iqbal,
Rahul Krishna,
Mohammad Ali Javidian,
Baishakhi Ray,
Pooyan Jamshidi
Abstract:
Modern computer systems are highly configurable, with the total variability space sometimes larger than the number of atoms in the universe. Understanding and reasoning about the performance behavior of highly configurable systems, over a vast and variable space, is challenging. State-of-the-art methods for performance modeling and analyses rely on predictive machine learning models, therefore, th…
▽ More
Modern computer systems are highly configurable, with the total variability space sometimes larger than the number of atoms in the universe. Understanding and reasoning about the performance behavior of highly configurable systems, over a vast and variable space, is challenging. State-of-the-art methods for performance modeling and analyses rely on predictive machine learning models, therefore, they become (i) unreliable in unseen environments (e.g., different hardware, workloads), and (ii) may produce incorrect explanations. To tackle this, we propose a new method, called Unicorn, which (i) captures intricate interactions between configuration options across the software-hardware stack and (ii) describes how such interactions can impact performance variations via causal inference. We evaluated Unicorn on six highly configurable systems, including three on-device machine learning systems, a video encoder, a database management system, and a data analytics pipeline. The experimental results indicate that Unicorn outperforms state-of-the-art performance debugging and optimization methods in finding effective repairs for performance faults and finding configurations with near-optimal performance. Further, unlike the existing methods, the learned causal performance models reliably predict performance for new environments.
△ Less
Submitted 17 March, 2022; v1 submitted 20 January, 2022;
originally announced January 2022.
-
Coconut trees detection and segmentation in aerial imagery using mask region-based convolution neural network
Authors:
Muhammad Shakaib Iqbal,
Hazrat Ali,
Son N. Tran,
Talha Iqbal
Abstract:
Food resources face severe damages under extraordinary situations of catastrophes such as earthquakes, cyclones, and tsunamis. Under such scenarios, speedy assessment of food resources from agricultural land is critical as it supports aid activity in the disaster hit areas. In this article, a deep learning approach is presented for the detection and segmentation of coconut tress in aerial imagery…
▽ More
Food resources face severe damages under extraordinary situations of catastrophes such as earthquakes, cyclones, and tsunamis. Under such scenarios, speedy assessment of food resources from agricultural land is critical as it supports aid activity in the disaster hit areas. In this article, a deep learning approach is presented for the detection and segmentation of coconut tress in aerial imagery provided through the AI competition organized by the World Bank in collaboration with OpenAerialMap and WeRobotics. Maked Region-based Convolutional Neural Network approach was used identification and segmentation of coconut trees. For the segmentation task, Mask R-CNN model with ResNet50 and ResNet1010 based architectures was used. Several experiments with different configuration parameters were performed and the best configuration for the detection of coconut trees with more than 90% confidence factor was reported. For the purpose of evaluation, Microsoft COCO dataset evaluation metric namely mean average precision (mAP) was used. An overall 91% mean average precision for coconut trees detection was achieved.
△ Less
Submitted 10 May, 2021;
originally announced May 2021.
-
A Survey of Methods for Managing the Classification and Solution of Data Imbalance Problem
Authors:
Khan Md. Hasib,
Md. Sadiq Iqbal,
Faisal Muhammad Shah,
Jubayer Al Mahmud,
Mahmudul Hasan Popel,
Md. Imran Hossain Showrov,
Shakil Ahmed,
Obaidur Rahman
Abstract:
The problem of class imbalance is extensive for focusing on numerous applications in the real world. In such a situation, nearly all of the examples are labeled as one class called majority class, while far fewer examples are labeled as the other class usually, the more important class is called minority. Over the last few years, several types of research have been carried out on the issue of clas…
▽ More
The problem of class imbalance is extensive for focusing on numerous applications in the real world. In such a situation, nearly all of the examples are labeled as one class called majority class, while far fewer examples are labeled as the other class usually, the more important class is called minority. Over the last few years, several types of research have been carried out on the issue of class imbalance, including data sampling, cost-sensitive analysis, Genetic Programming based models, bagging, boosting, etc. Nevertheless, in this survey paper, we enlisted the 24 related studies in the years 2003, 2008, 2010, 2012 and 2014 to 2019, focusing on the architecture of single, hybrid, and ensemble method design to understand the current status of improving classification output in machine learning techniques to fix problems with class imbalances. This survey paper also includes a statistical analysis of the classification algorithms under various methods and several other experimental conditions, as well as datasets used in different research papers.
△ Less
Submitted 22 December, 2020;
originally announced December 2020.
-
CADET: Debugging and Fixing Misconfigurations using Counterfactual Reasoning
Authors:
Rahul Krishna,
Md Shahriar Iqbal,
Mohammad Ali Javidian,
Baishakhi Ray,
Pooyan Jamshidi
Abstract:
Modern computing platforms are highly-configurable with thousands of interacting configurations. However, configuring these systems is challenging. Erroneous configurations can cause unexpected non-functional faults. This paper proposes CADET (short for Causal Debugging Toolkit) that enables users to identify, explain, and fix the root cause of non-functional faults early and in a principled fashi…
▽ More
Modern computing platforms are highly-configurable with thousands of interacting configurations. However, configuring these systems is challenging. Erroneous configurations can cause unexpected non-functional faults. This paper proposes CADET (short for Causal Debugging Toolkit) that enables users to identify, explain, and fix the root cause of non-functional faults early and in a principled fashion. CADET builds a causal model by observing the performance of the system under different configurations. Then, it uses casual path extraction followed by counterfactual reasoning over the causal model to: (a) identify the root causes of non-functional faults, (b) estimate the effects of various configurable parameters on the performance objective(s), and (c) prescribe candidate repairs to the relevant configuration options to fix the non-functional fault. We evaluated CADET on 5 highly-configurable systems deployed on 3 NVIDIA Jetson systems-on-chip. We compare CADET with state-of-the-art configuration optimization and ML-based debugging approaches. The experimental results indicate that CADET can find effective repairs for faults in multiple non-functional properties with (at most) 17% more accuracy, 28% higher gain, and $40\times$ speed-up than other ML-based performance debugging methods. Compared to multi-objective optimization approaches, CADET can find fixes (at most) $9\times$ faster with comparable or better performance gain. Our case study of non-functional faults reported in NVIDIA's forum show that CADET can find $14%$ better repairs than the experts' advice in less than 30 minutes.
△ Less
Submitted 8 March, 2021; v1 submitted 12 October, 2020;
originally announced October 2020.
-
An Environmentally Sustainable Closed-Loop Supply Chain Network Design under Uncertainty: Application of Optimization
Authors:
Md. Mohsin Ahmed,
S. M. Salauddin Iqbal,
Tazrin Jahan Priyanka,
Mohammad Arani,
Mohsen Momenitabar,
Md Mashum Billal
Abstract:
Newly, the rates of energy and material consumption to augment industrial pro-duction are substantially high, thus the environmentally sustainable industrial de-velopment has emerged as the main issue of either developed or develo** coun-tries. A novel approach to supply chain management is proposed to maintain economic growth along with environmentally friendly concerns for the design of the su…
▽ More
Newly, the rates of energy and material consumption to augment industrial pro-duction are substantially high, thus the environmentally sustainable industrial de-velopment has emerged as the main issue of either developed or develo** coun-tries. A novel approach to supply chain management is proposed to maintain economic growth along with environmentally friendly concerns for the design of the supply chain network. In this paper, a new green supply chain design approach has been suggested to maintain the financial virtue accompanying the environ-mental factors that required to be mitigated the negative effect of rapid industrial development on the environment. This approach has been suggested a multi-objective mathematical model minimizing the total costs and CO2 emissions for establishing an environmentally sustainable closed-loop supply chain. Two opti-mization methods are used namely Epsilon Constraint Method, and Genetic Al-gorithm Optimization Method. The results of the two mentioned methods have been compared and illustrated their effectiveness. The outcome of the analysis is approved to verify the accuracy of the proposed model to deal with financial and environmental issues concurrently.
△ Less
Submitted 24 September, 2020;
originally announced September 2020.
-
Minimum Length Scheduling for Discrete-Rate Full-Duplex Wireless Powered Communication Networks
Authors:
Muhammad Shahid Iqbal,
Yalcin Sadi,
Sinem Coleri
Abstract:
In this paper, we consider a wireless powered communication network where multiple users with RF energy harvesting capabilities communicate to a hybrid energy and information access point (HAP) in full-duplex mode. Each user has to transmit a certain amount of data with a transmission rate from a finite set of discrete rate levels, using the energy initially available in its battery and the energy…
▽ More
In this paper, we consider a wireless powered communication network where multiple users with RF energy harvesting capabilities communicate to a hybrid energy and information access point (HAP) in full-duplex mode. Each user has to transmit a certain amount of data with a transmission rate from a finite set of discrete rate levels, using the energy initially available in its battery and the energy it can harvest until the end of its transmission. Considering this model, we propose a novel discrete rate based minimum length scheduling problem to determine the optimal power control, rate adaptation and transmission schedule subject to data, energy causality and maximum transmit power constraints. The proposed optimization problem is proven to be NP-hard which requires exponential-time algorithms to solve for the global optimum. As a solution strategy, first, we demonstrate that the power control and rate adaptation, and scheduling problems can be solved separately in the optimal solution. For the power control and rate adaptation problem, we derive the optimal solution based on the proposed minimum length scheduling slot definition. For the scheduling, we classify the problem based on the distribution of minimum length scheduling slots of the users over time. For the non-overlap** slots scenario, we present the optimal scheduling algorithm. For the overlap** scenario, we propose a polynomial-time heuristic scheduling algorithm.
△ Less
Submitted 13 February, 2020;
originally announced February 2020.
-
Minimum Length Scheduling for Full Duplex Time-Critical Wireless Powered Communication Networks
Authors:
Muhammad Shahid Iqbal,
Yalcin Sadi,
Sinem Coleri
Abstract:
Radio frequency (RF) energy harvesting is key in attaining perpetual lifetime for time-critical wireless powered communication networks due to full control on energy transfer, far field region, small and low-cost circuitry. In this paper, we propose a novel minimum length scheduling problem to determine the optimal power control, time allocation and transmission schedule subject to data, energy ca…
▽ More
Radio frequency (RF) energy harvesting is key in attaining perpetual lifetime for time-critical wireless powered communication networks due to full control on energy transfer, far field region, small and low-cost circuitry. In this paper, we propose a novel minimum length scheduling problem to determine the optimal power control, time allocation and transmission schedule subject to data, energy causality and maximum transmit power constraints in a full-duplex wireless powered communication network. We first formulate the problem as a mixed integer non-linear programming problem and conjecture that the problem is NP-hard. As a solution strategy, we demonstrate that the power control and time allocation, and scheduling problems can be solved separately in the optimal solution. For the power control and time allocation problem, we derive the optimal solution by using Karush-Kuhn-Tucker conditions. For the scheduling, we introduce a penalty function allowing reformulation as a sum penalty minimization problem. Upon derivation of the optimality conditions based on the characteristics of the penalty function, we propose two polynomial-time heuristic algorithms and a reduced-complexity exact algorithm employing smart pruning techniques. Via extensive simulations, we illustrate that the proposed heuristic schemes outperform the previously proposed schemes for predetermined transmission order of users and achieve close-to-optimal solutions.
△ Less
Submitted 21 June, 2020; v1 submitted 3 February, 2020;
originally announced February 2020.
-
FlexiBO: A Decoupled Cost-Aware Multi-Objective Optimization Approach for Deep Neural Networks
Authors:
Md Shahriar Iqbal,
Jianhai Su,
Lars Kotthoff,
Pooyan Jamshidi
Abstract:
The design of machine learning systems often requires trading off different objectives, for example, prediction error and energy consumption for deep neural networks (DNNs). Typically, no single design performs well in all objectives; therefore, finding Pareto-optimal designs is of interest. The search for Pareto-optimal designs involves evaluating designs in an iterative process, and the measurem…
▽ More
The design of machine learning systems often requires trading off different objectives, for example, prediction error and energy consumption for deep neural networks (DNNs). Typically, no single design performs well in all objectives; therefore, finding Pareto-optimal designs is of interest. The search for Pareto-optimal designs involves evaluating designs in an iterative process, and the measurements are used to evaluate an acquisition function that guides the search process. However, measuring different objectives incurs different costs. For example, the cost of measuring the prediction error of DNNs is orders of magnitude higher than that of measuring the energy consumption of a pre-trained DNN, as it requires re-training the DNN. Current state-of-the-art methods do not consider this difference in objective evaluation cost, potentially incurring expensive evaluations of objective functions in the optimization process. In this paper, we develop a novel decoupled and cost-aware multi-objective optimization algorithm, we call Flexible Multi-Objective Bayesian Optimization (FlexiBO) to address this issue. FlexiBO weights the improvement of the hypervolume of the Pareto region by the measurement cost of each objective to balance the expense of collecting new information with the knowledge gained through objective evaluations, preventing us from performing expensive measurements for little to no gain. We evaluate FlexiBO on seven state-of-the-art DNNs for image recognition, natural language processing (NLP), and speech-to-text translation. Our results indicate that, given the same total experimental budget, FlexiBO discovers designs with 4.8$\%$ to 12.4$\%$ lower hypervolume error than the best method in state-of-the-art multi-objective optimization.
△ Less
Submitted 21 August, 2022; v1 submitted 17 January, 2020;
originally announced January 2020.
-
Throughput Maximization for Full Duplex Wireless Powered Communication Networks
Authors:
Muhammad Shahid Iqbal,
Yalcin Sadi,
Sinem Coleri
Abstract:
In this paper, we consider a full duplex wireless powered communication network where multiple users with RF energy harvesting capabilities communicate to a hybrid energy and information access point. An optimization framework is proposed with the objective of maximizing the sum throughput of the users subject to energy causality and maximum transmit power constraints considering a realistic energ…
▽ More
In this paper, we consider a full duplex wireless powered communication network where multiple users with RF energy harvesting capabilities communicate to a hybrid energy and information access point. An optimization framework is proposed with the objective of maximizing the sum throughput of the users subject to energy causality and maximum transmit power constraints considering a realistic energy harvesting model incorporating initial battery levels of the users. The joint optimization of power control, time allocation and scheduling is mathematically formulated as a mixed integer non linear programming problem which is hard to solve for a global optimum. The optimal power and time allocation and scheduling decisions are investigated separately based on the optimality analysis on the optimization variables. Optimal power and time allocation problem is proven to be convex for a given transmission order. Based on the derived optimality conditions, we propose a fast polynomial-time complexity heuristic algorithm. We illustrate that the proposed algorithm performs very close-to-optimal while significantly outperforming an equal time allocation based scheduling scheme.
△ Less
Submitted 29 October, 2019;
originally announced October 2019.
-
Transfer Learning for Performance Modeling of Deep Neural Network Systems
Authors:
Md Shahriar Iqbal,
Lars Kotthoff,
Pooyan Jamshidi
Abstract:
Modern deep neural network (DNN) systems are highly configurable with large a number of options that significantly affect their non-functional behavior, for example inference time and energy consumption. Performance models allow to understand and predict the effects of such configuration options on system behavior, but are costly to build because of large configuration spaces. Performance models f…
▽ More
Modern deep neural network (DNN) systems are highly configurable with large a number of options that significantly affect their non-functional behavior, for example inference time and energy consumption. Performance models allow to understand and predict the effects of such configuration options on system behavior, but are costly to build because of large configuration spaces. Performance models from one environment cannot be transferred directly to another; usually models are rebuilt from scratch for different environments, for example different hardware. Recently, transfer learning methods have been applied to reuse knowledge from performance models trained in one environment in another. In this paper, we perform an empirical study to understand the effectiveness of different transfer learning strategies for building performance models of DNN systems. Our results show that transferring information on the most influential configuration options and their interactions is an effective way of reducing the cost to build performance models in new environments.
△ Less
Submitted 4 April, 2019;
originally announced April 2019.
