-
Accelerating Transfer Learning with Near-Data Computation on Cloud Object Stores
Authors:
Arsany Guirguis,
Diana Petrescu,
Florin Dinu,
Do Le Quoc,
Javier Picorel,
Rachid Guerraoui
Abstract:
Near-data computation techniques have been successfully deployed to mitigate the cloud network bottleneck between the storage and compute tiers. At Huawei, we are currently looking to get more value from these techniques by broadening their applicability. Machine learning (ML) applications are an appealing and timely target. This paper describes our experience applying near-data computation techni…
▽ More
Near-data computation techniques have been successfully deployed to mitigate the cloud network bottleneck between the storage and compute tiers. At Huawei, we are currently looking to get more value from these techniques by broadening their applicability. Machine learning (ML) applications are an appealing and timely target. This paper describes our experience applying near-data computation techniques to transfer learning (TL), a widely popular ML technique, in the context of disaggregated cloud object stores. Our techniques benefit both cloud providers and users. They improve our operational efficiency while providing users the performance improvements they demand from us. The main practical challenge to consider is that the storage-side computational resources are limited. Our approach is to split the TL deep neural network (DNN) during the feature extraction phase, before the training phase. This reduces the network transfers to the compute tier and further decouples the batch size of feature extraction from the training batch size. This facilitates our second technique, storage-side batch adaptation, which enables increased concurrency in the storage tier while avoiding out-of-memory errors. Guided by these insights, we present HAPI, our processing system for TL that spans the compute and storage tiers while remaining transparent to the user. Our evaluation with several state-of-the-art DNNs, such as ResNet, VGG, and Transformer, shows up to 11x improvement in application runtime and up to 8.3x reduction in the data transferred from the storage to the compute tier compared to running the computation entirely in the compute tier.
△ Less
Submitted 9 January, 2023; v1 submitted 16 October, 2022;
originally announced October 2022.
-
Understanding and taming SSD read performance variability: HDFS case study
Authors:
María F. Borge,
Florin Dinu,
Willy Zwaenepoel
Abstract:
In this paper we analyze the influence that lower layers (file system, OS, SSD) have on HDFS' ability to extract maximum performance from SSDs on the read path. We uncover and analyze three surprising performance slowdowns induced by lower layers that result in HDFS read throughput loss. First, intrinsic slowdown affects reads from every new file system extent for a variable amount of time. Second…
▽ More
In this paper we analyze the influence that lower layers (file system, OS, SSD) have on HDFS' ability to extract maximum performance from SSDs on the read path. We uncover and analyze three surprising performance slowdowns induced by lower layers that result in HDFS read throughput loss. First, intrinsic slowdown affects reads from every new file system extent for a variable amount of time. Second, temporal slowdown appears temporarily and periodically and is workload-agnostic. Third, in permanent slowdown, some files can individually and permanently become slower after a period of time. We analyze the impact of these slowdowns on HDFS and show significant throughput loss. Individually, each of the slowdowns can cause a read throughput loss of 10-15%. However, their effect is cumulative. When all slowdowns happen concurrently, read throughput drops by as much as 30%. We further analyze mitigation techniques and show that two of the three slowdowns could be addressed via increased IO request parallelism in the lower layers. Unfortunately, HDFS cannot automatically adapt to use such additional parallelism. Our results point to a need for adaptability in storage stacks. The reason is that an access pattern that maximizes performance in the common case is not necessarily the same one that can mask performance fluctuations.
△ Less
Submitted 21 March, 2019;
originally announced March 2019.
-
Don't cry over spilled records: Memory elasticity of data-parallel applications and its application to cluster scheduling
Authors:
Calin Iorgulescu,
Florin Dinu,
Aunn Raza,
Wajih Ul Hassan,
Willy Zwaenepoel
Abstract:
Understanding the performance of data-parallel workloads when resource-constrained has significant practical importance but unfortunately has received only limited attention. This paper identifies, quantifies and demonstrates memory elasticity, an intrinsic property of data-parallel tasks. Memory elasticity allows tasks to run with significantly less memory that they would ideally want while only…
▽ More
Understanding the performance of data-parallel workloads when resource-constrained has significant practical importance but unfortunately has received only limited attention. This paper identifies, quantifies and demonstrates memory elasticity, an intrinsic property of data-parallel tasks. Memory elasticity allows tasks to run with significantly less memory that they would ideally want while only paying a moderate performance penalty. For example, we find that given as little as 10% of ideal memory, PageRank and NutchIndexing Hadoop reducers become only 1.2x/1.75x and 1.08x slower. We show that memory elasticity is prevalent in the Hadoop, Spark, Tez and Flink frameworks. We also show that memory elasticity is predictable in nature by building simple models for Hadoop and extending them to Tez and Spark.
To demonstrate the potential benefits of leveraging memory elasticity, this paper further explores its application to cluster scheduling. In this setting, we observe that the resource vs. time trade-off enabled by memory elasticity becomes a task queuing time vs task runtime trade-off. Tasks may complete faster when scheduled with less memory because their waiting time is reduced. We show that a scheduler can turn this task-level trade-off into improved job completion time and cluster-wide memory utilization. We have integrated memory elasticity into Apache YARN. We show gains of up to 60% in average job completion time on a 50-node Hadoop cluster. Extensive simulations show similar improvements over a large number of scenarios.
△ Less
Submitted 14 February, 2017;
originally announced February 2017.
