-
Accelerating DNN Training with Structured Data Gradient Pruning
Authors:
Bradley McDanel,
Helia Dinh,
John Magallanes
Abstract:
Weight pruning is a technique to make Deep Neural Network (DNN) inference more computationally efficient by reducing the number of model parameters over the course of training. However, most weight pruning techniques generally does not speed up DNN training and can even require more iterations to reach model convergence. In this work, we propose a novel Structured Data Gradient Pruning (SDGP) meth…
▽ More
Weight pruning is a technique to make Deep Neural Network (DNN) inference more computationally efficient by reducing the number of model parameters over the course of training. However, most weight pruning techniques generally does not speed up DNN training and can even require more iterations to reach model convergence. In this work, we propose a novel Structured Data Gradient Pruning (SDGP) method that can speed up training without impacting model convergence. This approach enforces a specific sparsity structure, where only N out of every M elements in a matrix can be nonzero, making it amenable to hardware acceleration. Modern accelerators such as the Nvidia A100 GPU support this type of structured sparsity for 2 nonzeros per 4 elements in a reduction. Assuming hardware support for 2:4 sparsity, our approach can achieve a 15-25\% reduction in total training time without significant impact to performance. Source code and pre-trained models are available at \url{https://github.com/BradMcDanel/sdgp}.
△ Less
Submitted 1 February, 2022;
originally announced February 2022.
-
Sequen-C: A Multilevel Overview of Temporal Event Sequences
Authors:
Jessica Magallanes,
Tony Stone,
Paul D Morris,
Suzanne Mason,
Steven Wood,
Maria-Cruz Villa-Uriol
Abstract:
Building a visual overview of temporal event sequences with an optimal level-of-detail (i.e. simplified but informative) is an ongoing challenge - expecting the user to zoom into every important aspect of the overview can lead to missing insights. We propose a technique to build a multilevel overview of event sequences, whose granularity can be transformed across sequence clusters (vertical level-…
▽ More
Building a visual overview of temporal event sequences with an optimal level-of-detail (i.e. simplified but informative) is an ongoing challenge - expecting the user to zoom into every important aspect of the overview can lead to missing insights. We propose a technique to build a multilevel overview of event sequences, whose granularity can be transformed across sequence clusters (vertical level-of-detail) or longitudinally (horizontal level-of-detail), using hierarchical aggregation and a novel cluster data representation Align-Score-Simplify. By default, the overview shows an optimal number of sequence clusters obtained through the average silhouette width metric - then users are able to explore alternative optimal sequence clusterings. The vertical level-of-detail of the overview changes along with the number of clusters, whilst the horizontal level-of-detail refers to the level of summarization applied to each cluster representation. The proposed technique has been implemented into a visualization system called Sequence Cluster Explorer (Sequen-C) that allows multilevel and detail-on-demand exploration through three coordinated views, and the inspection of data attributes at cluster, unique sequence, and individual sequence level. We present two case studies using real-world datasets in the healthcare domain: CUREd and MIMIC-III; which demonstrate how the technique can aid users to obtain a summary of common and deviating pathways, and explore data attributes for selected patterns.
△ Less
Submitted 6 August, 2021;
originally announced August 2021.
-
Analyzing Time Attributes in Temporal Event Sequences
Authors:
Jessica Magallanes,
Lindsey van Gemeren,
Steven Wood,
Maria-Cruz Villa-Uriol
Abstract:
Event data is present in a variety of domains such as electronic health records, daily living activities and web clickstream records. Current visualization methods to explore event data focus on discovering sequential patterns but present limitations when studying time attributes in event sequences. Time attributes are especially important when studying waiting times or lengths of visit in patient…
▽ More
Event data is present in a variety of domains such as electronic health records, daily living activities and web clickstream records. Current visualization methods to explore event data focus on discovering sequential patterns but present limitations when studying time attributes in event sequences. Time attributes are especially important when studying waiting times or lengths of visit in patient flow analysis. We propose a visual analytics methodology that allows the identification of trends and outliers in respect of duration and time of occurrence in event sequences. The proposed method presents event data using a single Sequential and Time Patterns overview. User-driven alignment by multiple events, sorting by sequence similarity and a novel visual encoding of events allows the comparison of time trends across and within sequences. The proposed visualization allows the derivation of findings that otherwise could not be obtained using traditional visualizations. The proposed methodology has been applied to a real-world dataset provided by Sheffield Teaching Hospitals NHS Foundation Trust, for which four classes of conclusions were derived.
△ Less
Submitted 2 August, 2019;
originally announced August 2019.
