-
Quantum Computing: Vision and Challenges
Authors:
Sukhpal Singh Gill,
Oktay Cetinkaya,
Stefano Marrone,
Daniel Claudino,
David Haunschild,
Leon Schlote,
Huaming Wu,
Carlo Ottaviani,
Xiaoyuan Liu,
Sree Pragna Machupalli,
Kamalpreet Kaur,
Priyansh Arora,
Ji Liu,
Ahmed Farouk,
Houbing Herbert Song,
Steve Uhlig,
Kotagiri Ramamohanarao
Abstract:
The recent development of quantum computing, which uses entanglement, superposition, and other quantum fundamental concepts, can provide substantial processing advantages over traditional computing. These quantum features help solve many complex problems that cannot be solved with conventional computing methods. These problems include modeling quantum mechanics, logistics, chemical-based advances,…
▽ More
The recent development of quantum computing, which uses entanglement, superposition, and other quantum fundamental concepts, can provide substantial processing advantages over traditional computing. These quantum features help solve many complex problems that cannot be solved with conventional computing methods. These problems include modeling quantum mechanics, logistics, chemical-based advances, drug design, statistical science, sustainable energy, banking, reliable communication, and quantum chemical engineering. The last few years have witnessed remarkable advancements in quantum software and algorithm creation and quantum hardware research, which has significantly advanced the prospect of realizing quantum computers. It would be helpful to have comprehensive literature research on this area to grasp the current status and find outstanding problems that require considerable attention from the research community working in the quantum computing industry. To better understand quantum computing, this paper examines the foundations and vision based on current research in this area. We discuss cutting-edge developments in quantum computer hardware advancement and subsequent advances in quantum cryptography, quantum software, and high-scalability quantum computers. Many potential challenges and exciting new trends for quantum technology research and development are highlighted in this paper for a broader debate.
△ Less
Submitted 5 June, 2024; v1 submitted 4 March, 2024;
originally announced March 2024.
-
Modern Computing: Vision and Challenges
Authors:
Sukhpal Singh Gill,
Huaming Wu,
Panos Patros,
Carlo Ottaviani,
Priyansh Arora,
Victor Casamayor Pujol,
David Haunschild,
Ajith Kumar Parlikad,
Oktay Cetinkaya,
Hanan Lutfiyya,
Vlado Stankovski,
Ruidong Li,
Yuemin Ding,
Junaid Qadir,
Ajith Abraham,
Soumya K. Ghosh,
Houbing Herbert Song,
Rizos Sakellariou,
Omer Rana,
Joel J. P. C. Rodrigues,
Salil S. Kanhere,
Schahram Dustdar,
Steve Uhlig,
Kotagiri Ramamohanarao,
Rajkumar Buyya
Abstract:
Over the past six decades, the computing systems field has experienced significant transformations, profoundly impacting society with transformational developments, such as the Internet and the commodification of computing. Underpinned by technological advancements, computer systems, far from being static, have been continuously evolving and adapting to cover multifaceted societal niches. This has…
▽ More
Over the past six decades, the computing systems field has experienced significant transformations, profoundly impacting society with transformational developments, such as the Internet and the commodification of computing. Underpinned by technological advancements, computer systems, far from being static, have been continuously evolving and adapting to cover multifaceted societal niches. This has led to new paradigms such as cloud, fog, edge computing, and the Internet of Things (IoT), which offer fresh economic and creative opportunities. Nevertheless, this rapid change poses complex research challenges, especially in maximizing potential and enhancing functionality. As such, to maintain an economical level of performance that meets ever-tighter requirements, one must understand the drivers of new model emergence and expansion, and how contemporary challenges differ from past ones. To that end, this article investigates and assesses the factors influencing the evolution of computing systems, covering established systems and architectures as well as newer developments, such as serverless computing, quantum computing, and on-device AI on edge devices. Trends emerge when one traces technological trajectory, which includes the rapid obsolescence of frameworks due to business and technical constraints, a move towards specialized systems and models, and varying approaches to centralized and decentralized control. This comprehensive review of modern computing systems looks ahead to the future of research in the field, highlighting key challenges and emerging trends, and underscoring their importance in cost-effectively driving technological progress.
△ Less
Submitted 4 January, 2024;
originally announced January 2024.
-
Strategic Decisions Survey, Taxonomy, and Future Directions from Artificial Intelligence Perspective
Authors:
Caesar Wu,
Kotagiri Ramamohanarao,
Rui Zhang,
Pascal Bouvry
Abstract:
Strategic Decision-Making is always challenging because it is inherently uncertain, ambiguous, risky, and complex. It is the art of possibility. We develop a systematic taxonomy of decision-making frames that consists of 6 bases, 18 categorical, and 54 frames. We aim to lay out the computational foundation that is possible to capture a comprehensive landscape view of a strategic problem. Compared…
▽ More
Strategic Decision-Making is always challenging because it is inherently uncertain, ambiguous, risky, and complex. It is the art of possibility. We develop a systematic taxonomy of decision-making frames that consists of 6 bases, 18 categorical, and 54 frames. We aim to lay out the computational foundation that is possible to capture a comprehensive landscape view of a strategic problem. Compared with traditional models, it covers irrational, non-rational and rational frames c dealing with certainty, uncertainty, complexity, ambiguity, chaos, and ignorance.
△ Less
Submitted 22 October, 2022;
originally announced October 2022.
-
Thermal Prediction for Efficient Energy Management of Clouds using Machine Learning
Authors:
Shashikant Ilager,
Kotagiri Ramamohanarao,
Rajkumar Buyya
Abstract:
Thermal management in the hyper-scale cloud data centers is a critical problem. Increased host temperature creates hotspots which significantly increases cooling cost and affects reliability. Accurate prediction of host temperature is crucial for managing the resources effectively. Temperature estimation is a non-trivial problem due to thermal variations in the data center. Existing solutions for…
▽ More
Thermal management in the hyper-scale cloud data centers is a critical problem. Increased host temperature creates hotspots which significantly increases cooling cost and affects reliability. Accurate prediction of host temperature is crucial for managing the resources effectively. Temperature estimation is a non-trivial problem due to thermal variations in the data center. Existing solutions for temperature estimation are inefficient due to their computational complexity and lack of accurate prediction. However, data-driven machine learning methods for temperature prediction is a promising approach. In this regard, we collect and study data from a private cloud and show the presence of thermal variations. We investigate several machine learning models to accurately predict the host temperature. Specifically, we propose a gradient boosting machine learning model for temperature prediction. The experiment results show that our model accurately predicts the temperature with the average RMSE value of 0.05 or an average prediction error of 2.38 degree Celsius, which is 6 degree Celsius less as compared to an existing theoretical model. In addition, we propose a dynamic scheduling algorithm to minimize the peak temperature of hosts. The results show that our algorithm reduces the peak temperature by 6.5 degree Celsius and consumes 34.5% less energy as compared to the baseline algorithm.
△ Less
Submitted 15 December, 2020; v1 submitted 6 November, 2020;
originally announced November 2020.
-
Segmented Pairwise Distance for Time Series with Large Discontinuities
Authors:
Jiabo He,
Sarah Erfani,
Sudanthi Wijewickrema,
Stephen O'Leary,
Kotagiri Ramamohanarao
Abstract:
Time series with large discontinuities are common in many scenarios. However, existing distance-based algorithms (e.g., DTW and its derivative algorithms) may perform poorly in measuring distances between these time series pairs. In this paper, we propose the segmented pairwise distance (SPD) algorithm to measure distances between time series with large discontinuities. SPD is orthogonal to distan…
▽ More
Time series with large discontinuities are common in many scenarios. However, existing distance-based algorithms (e.g., DTW and its derivative algorithms) may perform poorly in measuring distances between these time series pairs. In this paper, we propose the segmented pairwise distance (SPD) algorithm to measure distances between time series with large discontinuities. SPD is orthogonal to distance-based algorithms and can be embedded in them. We validate advantages of SPD-embedded algorithms over corresponding distance-based ones on both open datasets and a proprietary dataset of surgical time series (of surgeons performing a temporal bone surgery in a virtual reality surgery simulator). Experimental results demonstrate that SPD-embedded algorithms outperform corresponding distance-based ones in distance measurement between time series with large discontinuities, measured by the Silhouette index (SI).
△ Less
Submitted 23 September, 2020;
originally announced September 2020.
-
Learning Non-Unique Segmentation with Reward-Penalty Dice Loss
Authors:
Jiabo He,
Sarah Erfani,
Sudanthi Wijewickrema,
Stephen O'Leary,
Kotagiri Ramamohanarao
Abstract:
Semantic segmentation is one of the key problems in the field of computer vision, as it enables computer image understanding. However, most research and applications of semantic segmentation focus on addressing unique segmentation problems, where there is only one gold standard segmentation result for every input image. This may not be true in some problems, e.g., medical applications. We may have…
▽ More
Semantic segmentation is one of the key problems in the field of computer vision, as it enables computer image understanding. However, most research and applications of semantic segmentation focus on addressing unique segmentation problems, where there is only one gold standard segmentation result for every input image. This may not be true in some problems, e.g., medical applications. We may have non-unique segmentation annotations as different surgeons may perform successful surgeries for the same patient in slightly different ways. To comprehensively learn non-unique segmentation tasks, we propose the reward-penalty Dice loss (RPDL) function as the optimization objective for deep convolutional neural networks (DCNN). RPDL is capable of hel** DCNN learn non-unique segmentation by enhancing common regions and penalizing outside ones. Experimental results show that RPDL improves the performance of DCNN models by up to 18.4% compared with other loss functions on our collected surgical dataset.
△ Less
Submitted 23 September, 2020;
originally announced September 2020.
-
Dynamic Scheduling for Stochastic Edge-Cloud Computing Environments using A3C learning and Residual Recurrent Neural Networks
Authors:
Shreshth Tuli,
Shashikant Ilager,
Kotagiri Ramamohanarao,
Rajkumar Buyya
Abstract:
The ubiquitous adoption of Internet-of-Things (IoT) based applications has resulted in the emergence of the Fog computing paradigm, which allows seamlessly harnessing both mobile-edge and cloud resources. Efficient scheduling of application tasks in such environments is challenging due to constrained resource capabilities, mobility factors in IoT, resource heterogeneity, network hierarchy, and sto…
▽ More
The ubiquitous adoption of Internet-of-Things (IoT) based applications has resulted in the emergence of the Fog computing paradigm, which allows seamlessly harnessing both mobile-edge and cloud resources. Efficient scheduling of application tasks in such environments is challenging due to constrained resource capabilities, mobility factors in IoT, resource heterogeneity, network hierarchy, and stochastic behaviors. xisting heuristics and Reinforcement Learning based approaches lack generalizability and quick adaptability, thus failing to tackle this problem optimally. They are also unable to utilize the temporal workload patterns and are suitable only for centralized setups. However, Asynchronous-Advantage-Actor-Critic (A3C) learning is known to quickly adapt to dynamic scenarios with less data and Residual Recurrent Neural Network (R2N2) to quickly update model parameters. Thus, we propose an A3C based real-time scheduler for stochastic Edge-Cloud environments allowing decentralized learning, concurrently across multiple agents. We use the R2N2 architecture to capture a large number of host and task parameters together with temporal patterns to provide efficient scheduling decisions. The proposed model is adaptive and able to tune different hyper-parameters based on the application requirements. We explicate our choice of hyper-parameters through sensitivity analysis. The experiments conducted on real-world data set show a significant improvement in terms of energy consumption, response time, Service-Level-Agreement and running cost by 14.4%, 7.74%, 31.9%, and 4.64%, respectively when compared to the state-of-the-art algorithms.
△ Less
Submitted 1 September, 2020;
originally announced September 2020.
-
MurTree: Optimal Classification Trees via Dynamic Programming and Search
Authors:
Emir Demirović,
Anna Lukina,
Emmanuel Hebrard,
Jeffrey Chan,
James Bailey,
Christopher Leckie,
Kotagiri Ramamohanarao,
Peter J. Stuckey
Abstract:
Decision tree learning is a widely used approach in machine learning, favoured in applications that require concise and interpretable models. Heuristic methods are traditionally used to quickly produce models with reasonably high accuracy. A commonly criticised point, however, is that the resulting trees may not necessarily be the best representation of the data in terms of accuracy and size. In r…
▽ More
Decision tree learning is a widely used approach in machine learning, favoured in applications that require concise and interpretable models. Heuristic methods are traditionally used to quickly produce models with reasonably high accuracy. A commonly criticised point, however, is that the resulting trees may not necessarily be the best representation of the data in terms of accuracy and size. In recent years, this motivated the development of optimal classification tree algorithms that globally optimise the decision tree in contrast to heuristic methods that perform a sequence of locally optimal decisions. We follow this line of work and provide a novel algorithm for learning optimal classification trees based on dynamic programming and search. Our algorithm supports constraints on the depth of the tree and number of nodes. The success of our approach is attributed to a series of specialised techniques that exploit properties unique to classification trees. Whereas algorithms for optimal classification trees have traditionally been plagued by high runtimes and limited scalability, we show in a detailed experimental study that our approach uses only a fraction of the time required by the state-of-the-art and can handle datasets with tens of thousands of instances, providing several orders of magnitude improvements and notably contributing towards the practical realisation of optimal decision trees.
△ Less
Submitted 28 June, 2022; v1 submitted 24 July, 2020;
originally announced July 2020.
-
Application Management in Fog Computing Environments: A Taxonomy, Review and Future Directions
Authors:
Redowan Mahmud,
Kotagiri Ramamohanarao,
Rajkumar Buyya
Abstract:
The Internet of Things (IoT) paradigm is being rapidly adopted for the creation of smart environments in various domains. The IoT-enabled Cyber-Physical Systems (CPSs) associated with smart city, healthcare, Industry 4.0 and Agtech handle a huge volume of data and require data processing services from different types of applications in real-time. The Cloud-centric execution of IoT applications bar…
▽ More
The Internet of Things (IoT) paradigm is being rapidly adopted for the creation of smart environments in various domains. The IoT-enabled Cyber-Physical Systems (CPSs) associated with smart city, healthcare, Industry 4.0 and Agtech handle a huge volume of data and require data processing services from different types of applications in real-time. The Cloud-centric execution of IoT applications barely meets such requirements as the Cloud datacentres reside at a multi-hop distance from the IoT devices. \textit{Fog computing}, an extension of Cloud at the edge network, can execute these applications closer to data sources. Thus, Fog computing can improve application service delivery time and resist network congestion. However, the Fog nodes are highly distributed, heterogeneous and most of them are constrained in resources and spatial sharing. Therefore, efficient management of applications is necessary to fully exploit the capabilities of Fog nodes. In this work, we investigate the existing application management strategies in Fog computing and review them in terms of architecture, placement and maintenance. Additionally, we propose a comprehensive taxonomy and highlight the research gaps in Fog-based application management. We also discuss a perspective model and provide future research directions for further improvement of application management in Fog computing.
△ Less
Submitted 21 May, 2020;
originally announced May 2020.
-
Generative Image Inpainting with Submanifold Alignment
Authors:
Ang Li,
Jianzhong Qi,
Rui Zhang,
Xingjun Ma,
Kotagiri Ramamohanarao
Abstract:
Image inpainting aims at restoring missing regions of corrupted images, which has many applications such as image restoration and object removal. However, current GAN-based generative inpainting models do not explicitly exploit the structural or textural consistency between restored contents and their surrounding contexts.To address this limitation, we propose to enforce the alignment (or closenes…
▽ More
Image inpainting aims at restoring missing regions of corrupted images, which has many applications such as image restoration and object removal. However, current GAN-based generative inpainting models do not explicitly exploit the structural or textural consistency between restored contents and their surrounding contexts.To address this limitation, we propose to enforce the alignment (or closeness) between the local data submanifolds (or subspaces) around restored images and those around the original (uncorrupted) images during the learning process of GAN-based inpainting models. We exploit Local Intrinsic Dimensionality (LID) to measure, in deep feature space, the alignment between data submanifolds learned by a GAN model and those of the original data, from a perspective of both images (denoted as iLID) and local patches (denoted as pLID) of images. We then apply iLID and pLID as regularizations for GAN-based inpainting models to encourage two levels of submanifold alignment: 1) an image-level alignment for improving structural consistency, and 2) a patch-level alignment for improving textural details. Experimental results on four benchmark datasets show that our proposed model can generate more accurate results than state-of-the-art models.
△ Less
Submitted 12 September, 2020; v1 submitted 1 August, 2019;
originally announced August 2019.
-
A Differentially Private Algorithm for Range Queries on Trajectories
Authors:
Soheila Ghane,
Lars Kulik,
Kotagiri Ramamohanarao
Abstract:
We propose a novel algorithm to ensure $ε$-differential privacy for answering range queries on trajectory data. In order to guarantee privacy, differential privacy mechanisms add noise to either data or query, thus introducing errors to queries made and potentially decreasing the utility of information. In contrast to the state-of-the-art, our method achieves significantly lower error as it is the…
▽ More
We propose a novel algorithm to ensure $ε$-differential privacy for answering range queries on trajectory data. In order to guarantee privacy, differential privacy mechanisms add noise to either data or query, thus introducing errors to queries made and potentially decreasing the utility of information. In contrast to the state-of-the-art, our method achieves significantly lower error as it is the first data- and query-aware approach for such queries. The key challenge for answering range queries on trajectory data privately is to ensure an accurate count. Simply representing a trajectory as a set instead of \emph{sequence} of points will generally lead to highly inaccurate query answers as it ignores the sequential dependency of location points in trajectories, i.e., will violate the consistency of trajectory data. Furthermore, trajectories are generally unevenly distributed across a city and adding noise uniformly will generally lead to a poor utility. To achieve differential privacy, our algorithm adaptively adds noise to the input data according to the given query set. It first privately partitions the data space into uniform regions and computes the traffic density of each region. The regions and their densities, in addition to the given query set, are then used to estimate the distribution of trajectories over the queried space, which ensures high accuracy for the given query set. We show the accuracy and efficiency of our algorithm using extensive empirical evaluations on real and synthetic data sets.
△ Less
Submitted 18 July, 2019;
originally announced July 2019.
-
A Jointly Learned Context-Aware Place of Interest Embedding for Trip Recommendations
Authors:
Jiayuan He,
Jianzhong Qi,
Kotagiri Ramamohanarao
Abstract:
Trip recommendation is an important location-based service that helps relieve users from the time and efforts for trip planning. It aims to recommend a sequence of places of interest (POIs) for a user to visit that maximizes the user's satisfaction. When adding a POI to a recommended trip, it is essential to understand the context of the recommendation, including the POI popularity, other POIs co-…
▽ More
Trip recommendation is an important location-based service that helps relieve users from the time and efforts for trip planning. It aims to recommend a sequence of places of interest (POIs) for a user to visit that maximizes the user's satisfaction. When adding a POI to a recommended trip, it is essential to understand the context of the recommendation, including the POI popularity, other POIs co-occurring in the trip, and the preferences of the user. These contextual factors are learned separately in existing studies, while in reality, they impact jointly on a user's choice of a POI to visit. In this study, we propose a POI embedding model to jointly learn the impact of these contextual factors. We call the learned POI embedding a context-aware POI embedding. To showcase the effectiveness of this embedding, we apply it to generate trip recommendations given a user and a time budget. We propose two trip recommendation algorithms based on our context-aware POI embedding. The first algorithm finds the exact optimal trip by transforming and solving the trip recommendation problem as an integer linear programming problem. To achieve a high computation efficiency, the second algorithm finds a heuristically optimal trip based on adaptive large neighborhood search. We perform extensive experiments on real datasets. The results show that our proposed algorithms consistently outperform state-of-the-art algorithms in trip recommendation quality, with an advantage of up to 43% in F1-score.
△ Less
Submitted 24 August, 2018;
originally announced August 2018.
-
Performance-Aware Management of Cloud Resources: A Taxonomy and Future Directions
Authors:
Sara Kardani-Moghaddam,
Rajkumar Buyya,
Kotagiri Ramamohanarao
Abstract:
Dynamic nature of the cloud environment has made distributed resource management process a challenge for cloud service providers. The importance of maintaining the quality of service in accordance with customer expectations as well as the highly dynamic nature of cloud-hosted applications add new levels of complexity to the process. Advances to the big data learning approaches have shifted convent…
▽ More
Dynamic nature of the cloud environment has made distributed resource management process a challenge for cloud service providers. The importance of maintaining the quality of service in accordance with customer expectations as well as the highly dynamic nature of cloud-hosted applications add new levels of complexity to the process. Advances to the big data learning approaches have shifted conventional static capacity planning solutions to complex performance-aware resource management methods. It is shown that the process of decision making for resource adjustment is closely related to the behaviour of the system including the utilization of resources and application components. Therefore, a continuous monitoring of system attributes and performance metrics provide the raw data for the analysis of problems affecting the performance of the application. Data analytic methods such as statistical and machine learning approaches offer the required concepts, models and tools to dig into the data, find general rules, patterns and characteristics that define the functionality of the system. Obtained knowledge form the data analysis process helps to find out about the changes in the workloads, faulty components or problems that can cause system performance to degrade. A timely reaction to performance degradations can avoid violations of the service level agreements by performing proper corrective actions including auto-scaling or other resource adjustment solutions. In this paper, we investigate the main requirements and limitations in cloud resource management including a study of the approaches in workload and anomaly analysis in the context of the performance management in the cloud. A taxonomy of the works on this problem is presented which identifies the main approaches in existing researches from data analysis side to resource adjustment techniques.
△ Less
Submitted 7 August, 2018;
originally announced August 2018.
-
Learning with Bounded Instance- and Label-dependent Label Noise
Authors:
Jiacheng Cheng,
Tongliang Liu,
Kotagiri Ramamohanarao,
Dacheng Tao
Abstract:
Instance- and Label-dependent label Noise (ILN) widely exists in real-world datasets but has been rarely studied. In this paper, we focus on Bounded Instance- and Label-dependent label Noise (BILN), a particular case of ILN where the label noise rates -- the probabilities that the true labels of examples flip into the corrupted ones -- have upper bound less than $1$. Specifically, we introduce the…
▽ More
Instance- and Label-dependent label Noise (ILN) widely exists in real-world datasets but has been rarely studied. In this paper, we focus on Bounded Instance- and Label-dependent label Noise (BILN), a particular case of ILN where the label noise rates -- the probabilities that the true labels of examples flip into the corrupted ones -- have upper bound less than $1$. Specifically, we introduce the concept of distilled examples, i.e. examples whose labels are identical with the labels assigned for them by the Bayes optimal classifier, and prove that under certain conditions classifiers learnt on distilled examples will converge to the Bayes optimal classifier. Inspired by the idea of learning with distilled examples, we then propose a learning algorithm with theoretical guarantees for its robustness to BILN. At last, empirical evaluations on both synthetic and real-world datasets show effectiveness of our algorithm in learning with BILN.
△ Less
Submitted 10 December, 2020; v1 submitted 12 September, 2017;
originally announced September 2017.
-
A Technical Report: Entity Extraction using Both Character-based and Token-based Similarity
Authors:
Zeyi Wen,
Dong Deng,
Rui Zhang,
Kotagiri Ramamohanarao
Abstract:
Entity extraction is fundamental to many text mining tasks such as organisation name recognition. A popular approach to entity extraction is based on matching sub-string candidates in a document against a dictionary of entities. To handle spelling errors and name variations of entities, usually the matching is approximate and edit or Jaccard distance is used to measure dissimilarity between sub-st…
▽ More
Entity extraction is fundamental to many text mining tasks such as organisation name recognition. A popular approach to entity extraction is based on matching sub-string candidates in a document against a dictionary of entities. To handle spelling errors and name variations of entities, usually the matching is approximate and edit or Jaccard distance is used to measure dissimilarity between sub-string candidates and the entities. For approximate entity extraction from free text, existing work considers solely character-based or solely token-based similarity and hence cannot simultaneously deal with minor variations at token level and typos. In this paper, we address this problem by considering both character-based similarity and token-based similarity (i.e. two-level similarity). Measuring one-level (e.g. character-based) similarity is computationally expensive, and measuring two-level similarity is dramatically more expensive. By exploiting the properties of the two-level similarity and the weights of tokens, we develop novel techniques to significantly reduce the number of sub-string candidates that require computation of two-level similarity against the dictionary of entities. A comprehensive experimental study on real world datasets show that our algorithm can efficiently extract entities from documents and produce a high F1 score in the range of [0.91, 0.97].
△ Less
Submitted 12 February, 2017;
originally announced February 2017.
-
Big Data Analytics = Machine Learning + Cloud Computing
Authors:
Caesar Wu,
Rajkumar Buyya,
Kotagiri Ramamohanarao
Abstract:
Big Data can mean different things to different people. The scale and challenges of Big Data are often described using three attributes, namely Volume, Velocity and Variety (3Vs), which only reflect some of the aspects of data. In this chapter we review historical aspects of the term "Big Data" and the associated analytics. We augment 3Vs with additional attributes of Big Data to make it more comp…
▽ More
Big Data can mean different things to different people. The scale and challenges of Big Data are often described using three attributes, namely Volume, Velocity and Variety (3Vs), which only reflect some of the aspects of data. In this chapter we review historical aspects of the term "Big Data" and the associated analytics. We augment 3Vs with additional attributes of Big Data to make it more comprehensive and relevant. We show that Big Data is not just 3Vs, but 32 Vs, that is, 9 Vs covering the fundamental motivation behind Big Data, which is to incorporate Business Intelligence (BI) based on different hypothesis or statistical models so that Big Data Analytics (BDA) can enable decision makers to make useful predictions for some crucial decisions or researching results. History of Big Data has demonstrated that the most cost effective way of performing BDA is to employ Machine Learning (ML) on the Cloud Computing (CC) based infrastructure or simply, ML + CC -> BDA. This chapter is devoted to help decision makers by defining BDA as a solution and opportunity to address their business needs.
△ Less
Submitted 12 January, 2016;
originally announced January 2016.
-
Big Data Analytics-Enhanced Cloud Computing: Challenges, Architectural Elements, and Future Directions
Authors:
Rajkumar Buyya,
Kotagiri Ramamohanarao,
Chris Leckie,
Rodrigo N. Calheiros,
Amir Vahid Dastjerdi,
Steve Versteeg
Abstract:
The emergence of cloud computing has made dynamic provisioning of elastic capacity to applications on-demand. Cloud data centers contain thousands of physical servers hosting orders of magnitude more virtual machines that can be allocated on demand to users in a pay-as-you-go model. However, not all systems are able to scale up by just adding more virtual machines. Therefore, it is essential, even…
▽ More
The emergence of cloud computing has made dynamic provisioning of elastic capacity to applications on-demand. Cloud data centers contain thousands of physical servers hosting orders of magnitude more virtual machines that can be allocated on demand to users in a pay-as-you-go model. However, not all systems are able to scale up by just adding more virtual machines. Therefore, it is essential, even for scalable systems, to project workloads in advance rather than using a purely reactive approach. Given the scale of modern cloud infrastructures generating real time monitoring information, along with all the information generated by operating systems and applications, this data poses the issues of volume, velocity, and variety that are addressed by Big Data approaches. In this paper, we investigate how utilization of Big Data analytics helps in enhancing the operation of cloud computing environments. We discuss diverse applications of Big Data analytics in clouds, open issues for enhancing cloud operations via Big Data analytics, and architecture for anomaly detection and prevention in clouds along with future research directions.
△ Less
Submitted 22 October, 2015;
originally announced October 2015.
-
Tensor Canonical Correlation Analysis for Multi-view Dimension Reduction
Authors:
Yong Luo,
Dacheng Tao,
Yonggang Wen,
Kotagiri Ramamohanarao,
Chao Xu
Abstract:
Canonical correlation analysis (CCA) has proven an effective tool for two-view dimension reduction due to its profound theoretical foundation and success in practical applications. In respect of multi-view learning, however, it is limited by its capability of only handling data represented by two-view features, while in many real-world applications, the number of views is frequently many more. Alt…
▽ More
Canonical correlation analysis (CCA) has proven an effective tool for two-view dimension reduction due to its profound theoretical foundation and success in practical applications. In respect of multi-view learning, however, it is limited by its capability of only handling data represented by two-view features, while in many real-world applications, the number of views is frequently many more. Although the ad hoc way of simultaneously exploring all possible pairs of features can numerically deal with multi-view data, it ignores the high order statistics (correlation information) which can only be discovered by simultaneously exploring all features.
Therefore, in this work, we develop tensor CCA (TCCA) which straightforwardly yet naturally generalizes CCA to handle the data of an arbitrary number of views by analyzing the covariance tensor of the different views. TCCA aims to directly maximize the canonical correlation of multiple (more than two) views. Crucially, we prove that the multi-view canonical correlation maximization problem is equivalent to finding the best rank-1 approximation of the data covariance tensor, which can be solved efficiently using the well-known alternating least squares (ALS) algorithm. As a consequence, the high order correlation information contained in the different views is explored and thus a more reliable common subspace shared by all features can be obtained. In addition, a non-linear extension of TCCA is presented. Experiments on various challenge tasks, including large scale biometric structure prediction, internet advertisement classification and web image annotation, demonstrate the effectiveness of the proposed method.
△ Less
Submitted 8 February, 2015;
originally announced February 2015.
-
Decentralized Traffic Management Strategies for Sensor-Enabled Cars
Authors:
Ziyuan Wang,
Lars Kulik,
Kotagiri Ramamohanarao
Abstract:
Traffic Congestions and accidents are major concerns in today's transportation systems. This thesis investigates how to optimize traffic flow on highways, in particular for merging situations such as intersections where a ramp leads onto the highway. In our work, cars are equipped with sensors that can detect distance to neighboring cars, and communicate their velocity and acceleration readings…
▽ More
Traffic Congestions and accidents are major concerns in today's transportation systems. This thesis investigates how to optimize traffic flow on highways, in particular for merging situations such as intersections where a ramp leads onto the highway. In our work, cars are equipped with sensors that can detect distance to neighboring cars, and communicate their velocity and acceleration readings with one another. Sensor-enabled cars can locally exchange sensed information about the traffic and adapt their behavior much earlier than regular cars.
We propose proactive algorithms for merging different streams of sensor-enabled cars into a single stream. A proactive merging algorithm decouples the decision point from the actual merging point. Sensor-enabled cars allow us to decide where and when a car merges before it arrives at the actual merging point. This leads to a significant improvement in traffic flow as velocities can be adjusted appropriately. We compare proactive merging algorithms against the conventional priority-based merging algorithm in a controlled simulation environment. Experiment results show that proactive merging algorithms outperform the priority-based merging algorithm in terms of flow and delay.
△ Less
Submitted 18 June, 2009;
originally announced June 2009.
-
Lossy Bulk Synchronous Parallel Processing Model for Very Large Scale Grids
Authors:
Elankovan Sundararajan,
Aaron Harwood,
Kotagiri Ramamohanarao
Abstract:
The performance of a parallel algorithm in a very large scale grid is significantly influenced by the underlying Internet protocols and inter-connectivity. Many grid programming platforms use TCP due to its reliability, usually with some optimizations to reduce its costs. However, TCP does not perform well in a high bandwidth and high delay network environment. On the other hand, UDP is the fast…
▽ More
The performance of a parallel algorithm in a very large scale grid is significantly influenced by the underlying Internet protocols and inter-connectivity. Many grid programming platforms use TCP due to its reliability, usually with some optimizations to reduce its costs. However, TCP does not perform well in a high bandwidth and high delay network environment. On the other hand, UDP is the fastest protocol available because it omits connection setup process, acknowledgments and retransmissions sacrificing reliable transfer. Many new bulk data transfer schemes using UDP for data transmission such as RBUDP, Tsunami, and SABUL have been introduced and shown to have better performance compared to TCP. In this paper, we consider the use of UDP and examine the relationship between packet loss and speedup with respect to the number of grid nodes. Our measurement suggests that packet loss rates between 5%-15% on average are not uncommon between PlanetLab nodes that are widely distributed over the Internet. We show that transmitting multiple copies of same packet produces higher speedup. We show the minimum number of packet duplication required to maximize the possible speedup for a given number of nodes using a BSP based model. Our work demonstrates that by using an appropriate number of packet copies, we can increase performance of parallel program.
△ Less
Submitted 20 December, 2006; v1 submitted 19 November, 2006;
originally announced November 2006.
-
A Taxonomy of Data Grids for Distributed Data Sharing, Management and Processing
Authors:
Srikumar Venugopal,
Rajkumar Buyya,
Kotagiri Ramamohanarao
Abstract:
Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data shari…
▽ More
Data Grids have been adopted as the platform for scientific communities that need to share, access, transport, process and manage large data collections distributed worldwide. They combine high-end computing technologies with high-performance networking and wide-area storage management techniques. In this paper, we discuss the key concepts behind Data Grids and compare them with other data sharing and distribution paradigms such as content delivery networks, peer-to-peer networks and distributed databases. We then provide comprehensive taxonomies that cover various aspects of architecture, data transportation, data replication and resource allocation and scheduling. Finally, we map the proposed taxonomy to various Data Grid systems not only to validate the taxonomy but also to identify areas for future exploration. Through this taxonomy, we aim to categorise existing systems to better understand their goals and their methodology. This would help evaluate their applicability for solving similar problems. This taxonomy also provides a "gap analysis" of this area through which researchers can potentially identify new issues for investigation. Finally, we hope that the proposed taxonomy and map** also helps to provide an easy way for new practitioners to understand this complex area of research.
△ Less
Submitted 10 June, 2005;
originally announced June 2005.
