-
GenPIP: In-Memory Acceleration of Genome Analysis via Tight Integration of Basecalling and Read Map**
Authors:
Haiyu Mao,
Mohammed Alser,
Mohammad Sadrosadati,
Can Firtina,
Akanksha Baranwal,
Damla Senol Cali,
Aditya Manglik,
Nour Almadhoun Alserr,
Onur Mutlu
Abstract:
Nanopore sequencing is a widely-used high-throughput genome sequencing technology that can sequence long fragments of a genome into raw electrical signals at low cost. Nanopore sequencing requires two computationally-costly processing steps for accurate downstream genome analysis. The first step, basecalling, translates the raw electrical signals into nucleotide bases (i.e., A, C, G, T). The secon…
▽ More
Nanopore sequencing is a widely-used high-throughput genome sequencing technology that can sequence long fragments of a genome into raw electrical signals at low cost. Nanopore sequencing requires two computationally-costly processing steps for accurate downstream genome analysis. The first step, basecalling, translates the raw electrical signals into nucleotide bases (i.e., A, C, G, T). The second step, read map**, finds the correct location of a read in a reference genome. In existing genome analysis pipelines, basecalling and read map** are executed separately. We observe in this work that such separate execution of the two most time-consuming steps inherently leads to (1) significant data movement and (2) redundant computations on the data, slowing down the genome analysis pipeline. This paper proposes GenPIP, an in-memory genome analysis accelerator that tightly integrates basecalling and read map**. GenPIP improves the performance of the genome analysis pipeline with two key mechanisms: (1) in-memory fine-grained collaborative execution of the major genome analysis steps in parallel; (2) a new technique for early-rejection of low-quality and unmapped reads to timely stop the execution of genome analysis for such reads, reducing inefficient computation. Our experiments show that, for the execution of the genome analysis pipeline, GenPIP provides 41.6X (8.4X) speedup and 32.8X (20.8X) energy savings with negligible accuracy loss compared to the state-of-the-art software genome analysis tools executed on a state-of-the-art CPU (GPU). Compared to a design that combines state-of-the-art in-memory basecalling and read map** accelerators, GenPIP provides 1.39X speedup and 1.37X energy savings.
△ Less
Submitted 17 December, 2023; v1 submitted 18 September, 2022;
originally announced September 2022.
-
Scrooge: A Fast and Memory-Frugal Genomic Sequence Aligner for CPUs, GPUs, and ASICs
Authors:
Joël Lindegger,
Damla Senol Cali,
Mohammed Alser,
Juan Gómez-Luna,
Nika Mansouri Ghiasi,
Onur Mutlu
Abstract:
Pairwise sequence alignment is a very time-consuming step in common bioinformatics pipelines. Speeding up this step requires heuristics, efficient implementations, and/or hardware acceleration. A promising candidate for all of the above is the recently proposed GenASM algorithm. We identify and address three inefficiencies in the GenASM algorithm: it has a high amount of data movement, a large mem…
▽ More
Pairwise sequence alignment is a very time-consuming step in common bioinformatics pipelines. Speeding up this step requires heuristics, efficient implementations, and/or hardware acceleration. A promising candidate for all of the above is the recently proposed GenASM algorithm. We identify and address three inefficiencies in the GenASM algorithm: it has a high amount of data movement, a large memory footprint, and does some unnecessary work. We propose Scrooge, a fast and memory-frugal genomic sequence aligner. Scrooge includes three novel algorithmic improvements which reduce the data movement, memory footprint, and the number of operations in the GenASM algorithm. We provide efficient open-source implementations of the Scrooge algorithm for CPUs and GPUs, which demonstrate the significant benefits of our algorithmic improvements. For long reads, the CPU version of Scrooge achieves a 20.1x, 1.7x, and 2.1x speedup over KSW2, Edlib, and a CPU implementation of GenASM, respectively. The GPU version of Scrooge achieves a 4.0x 80.4x, 6.8x, 12.6x and 5.9x speedup over the CPU version of Scrooge, KSW2, Edlib, Darwin-GPU, and a GPU implementation of GenASM, respectively. We estimate an ASIC implementation of Scrooge to use 3.6x less chip area and 2.1x less power than a GenASM ASIC while maintaining the same throughput. Further, we systematically analyze the throughput and accuracy behavior of GenASM and Scrooge under various configurations. As the best configuration of Scrooge depends on the computing platform, we make several observations that can help guide future implementations of Scrooge. Availability: https://github.com/CMU-SAFARI/Scrooge
△ Less
Submitted 12 April, 2023; v1 submitted 21 August, 2022;
originally announced August 2022.
-
ApHMM: Accelerating Profile Hidden Markov Models for Fast and Energy-Efficient Genome Analysis
Authors:
Can Firtina,
Kamlesh Pillai,
Gurpreet S. Kalsi,
Bharathwaj Suresh,
Damla Senol Cali,
Jeremie Kim,
Taha Shahroodi,
Meryem Banu Cavlak,
Joel Lindegger,
Mohammed Alser,
Juan Gómez Luna,
Sreenivas Subramoney,
Onur Mutlu
Abstract:
Profile hidden Markov models (pHMMs) are widely employed in various bioinformatics applications to identify similarities between biological sequences, such as DNA or protein sequences. In pHMMs, sequences are represented as graph structures. These probabilities are subsequently used to compute the similarity score between a sequence and a pHMM graph. The Baum-Welch algorithm, a prevalent and highl…
▽ More
Profile hidden Markov models (pHMMs) are widely employed in various bioinformatics applications to identify similarities between biological sequences, such as DNA or protein sequences. In pHMMs, sequences are represented as graph structures. These probabilities are subsequently used to compute the similarity score between a sequence and a pHMM graph. The Baum-Welch algorithm, a prevalent and highly accurate method, utilizes these probabilities to optimize and compute similarity scores. However, the Baum-Welch algorithm is computationally intensive, and existing solutions offer either software-only or hardware-only approaches with fixed pHMM designs. We identify an urgent need for a flexible, high-performance, and energy-efficient HW/SW co-design to address the major inefficiencies in the Baum-Welch algorithm for pHMMs.
We introduce ApHMM, the first flexible acceleration framework designed to significantly reduce both computational and energy overheads associated with the Baum-Welch algorithm for pHMMs. ApHMM tackles the major inefficiencies in the Baum-Welch algorithm by 1) designing flexible hardware to accommodate various pHMM designs, 2) exploiting predictable data dependency patterns through on-chip memory with memoization techniques, 3) rapidly filtering out negligible computations using a hardware-based filter, and 4) minimizing redundant computations.
ApHMM achieves substantial speedups of 15.55x - 260.03x, 1.83x - 5.34x, and 27.97x when compared to CPU, GPU, and FPGA implementations of the Baum-Welch algorithm, respectively. ApHMM outperforms state-of-the-art CPU implementations in three key bioinformatics applications: 1) error correction, 2) protein family search, and 3) multiple sequence alignment, by 1.29x - 59.94x, 1.03x - 1.75x, and 1.03x - 1.95x, respectively, while improving their energy efficiency by 64.24x - 115.46x, 1.75x, 1.96x.
△ Less
Submitted 21 October, 2023; v1 submitted 20 July, 2022;
originally announced July 2022.
-
SeGraM: A Universal Hardware Accelerator for Genomic Sequence-to-Graph and Sequence-to-Sequence Map**
Authors:
Damla Senol Cali,
Konstantinos Kanellopoulos,
Joel Lindegger,
Zülal Bingöl,
Gurpreet S. Kalsi,
Ziyi Zuo,
Can Firtina,
Meryem Banu Cavlak,
Jeremie Kim,
Nika Mansouri Ghiasi,
Gagandeep Singh,
Juan Gómez-Luna,
Nour Almadhoun Alserr,
Mohammed Alser,
Sreenivas Subramoney,
Can Alkan,
Saugata Ghose,
Onur Mutlu
Abstract:
A critical step of genome sequence analysis is the map** of sequenced DNA fragments (i.e., reads) collected from an individual to a known linear reference genome sequence (i.e., sequence-to-sequence map**). Recent works replace the linear reference sequence with a graph-based representation of the reference genome, which captures the genetic variations and diversity across many individuals in…
▽ More
A critical step of genome sequence analysis is the map** of sequenced DNA fragments (i.e., reads) collected from an individual to a known linear reference genome sequence (i.e., sequence-to-sequence map**). Recent works replace the linear reference sequence with a graph-based representation of the reference genome, which captures the genetic variations and diversity across many individuals in a population. Map** reads to the graph-based reference genome (i.e., sequence-to-graph map**) results in notable quality improvements in genome analysis. Unfortunately, while sequence-to-sequence map** is well studied with many available tools and accelerators, sequence-to-graph map** is a more difficult computational problem, with a much smaller number of practical software tools currently available.
We analyze two state-of-the-art sequence-to-graph map** tools and reveal four key issues. We find that there is a pressing need to have a specialized, high-performance, scalable, and low-cost algorithm/hardware co-design that alleviates bottlenecks in both the seeding and alignment steps of sequence-to-graph map**.
To this end, we propose SeGraM, a universal algorithm/hardware co-designed genomic map** accelerator that can effectively and efficiently support both sequence-to-graph map** and sequence-to-sequence map**, for both short and long reads. To our knowledge, SeGraM is the first algorithm/hardware co-design for accelerating sequence-to-graph map**. SeGraM consists of two main components: (1) MinSeed, the first minimizer-based seeding accelerator; and (2) BitAlign, the first bitvector-based sequence-to-graph alignment accelerator.
We demonstrate that SeGraM provides significant improvements for multiple steps of the sequence-to-graph and sequence-to-sequence map** pipelines.
△ Less
Submitted 31 May, 2022; v1 submitted 12 May, 2022;
originally announced May 2022.
-
Algorithmic Improvement and GPU Acceleration of the GenASM Algorithm
Authors:
Joël Lindegger,
Damla Senol Cali,
Mohammed Alser,
Juan Gómez-Luna,
Onur Mutlu
Abstract:
We improve on GenASM, a recent algorithm for genomic sequence alignment, by significantly reducing its memory footprint and bandwidth requirement. Our algorithmic improvements reduce the memory footprint by 24$\times$ and the number of memory accesses by 12$\times$. We efficiently parallelize the algorithm for GPUs, achieving a 4.1$\times$ speedup over a CPU implementation of the same algorithm, a…
▽ More
We improve on GenASM, a recent algorithm for genomic sequence alignment, by significantly reducing its memory footprint and bandwidth requirement. Our algorithmic improvements reduce the memory footprint by 24$\times$ and the number of memory accesses by 12$\times$. We efficiently parallelize the algorithm for GPUs, achieving a 4.1$\times$ speedup over a CPU implementation of the same algorithm, a 62$\times$ speedup over minimap2's CPU-based KSW2 and a 7.2$\times$ speedup over the CPU-based Edlib for long reads.
△ Less
Submitted 28 March, 2022;
originally announced March 2022.
-
GenStore: A High-Performance and Energy-Efficient In-Storage Computing System for Genome Sequence Analysis
Authors:
Nika Mansouri Ghiasi,
Jisung Park,
Harun Mustafa,
Jeremie Kim,
Ataberk Olgun,
Arvid Gollwitzer,
Damla Senol Cali,
Can Firtina,
Haiyu Mao,
Nour Almadhoun Alserr,
Rachata Ausavarungnirun,
Nandita Vijaykumar,
Mohammed Alser,
Onur Mutlu
Abstract:
Read map** is a fundamental, yet computationally-expensive step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). To address the computational challenges in genome analysis, many prior works propose various approaches such as filters that select th…
▽ More
Read map** is a fundamental, yet computationally-expensive step in many genomics applications. It is used to identify potential matches and differences between fragments (called reads) of a sequenced genome and an already known genome (called a reference genome). To address the computational challenges in genome analysis, many prior works propose various approaches such as filters that select the reads that must undergo expensive computation, efficient heuristics, and hardware acceleration. While effective at reducing the computation overhead, all such approaches still require the costly movement of a large amount of data from storage to the rest of the system, which can significantly lower the end-to-end performance of read map** in conventional and emerging genomics systems.
We propose GenStore, the first in-storage processing system designed for genome sequence analysis that greatly reduces both data movement and computational overheads of genome sequence analysis by exploiting low-cost and accurate in-storage filters. GenStore leverages hardware/software co-design to address the challenges of in-storage processing, supporting reads with 1) different read lengths and error rates, and 2) different degrees of genetic variation. Through rigorous analysis of read map** processes, we meticulously design low-cost hardware accelerators and data/computation flows inside a NAND flash-based SSD. Our evaluation using a wide range of real genomic datasets shows that GenStore, when implemented in three modern SSDs, significantly improves the read map** performance of state-of-the-art software (hardware) baselines by 2.07-6.05$\times$ (1.52-3.32$\times$) for read sets with high similarity to the reference genome and 1.45-33.63$\times$ (2.70-19.2$\times$) for read sets with low similarity to the reference genome.
△ Less
Submitted 6 April, 2023; v1 submitted 21 February, 2022;
originally announced February 2022.
-
FastRemap: A Tool for Quickly Remap** Reads between Genome Assemblies
Authors:
Jeremie S. Kim,
Can Firtina,
Meryem Banu Cavlak,
Damla Senol Cali,
Can Alkan,
Onur Mutlu
Abstract:
A genome read data set can be quickly and efficiently remapped from one reference to another similar reference (e.g., between two reference versions or two similar species) using a variety of tools, e.g., the commonly-used CrossMap tool. With the explosion of available genomic data sets and references, high-performance remap** tools will be even more important for kee** up with the computation…
▽ More
A genome read data set can be quickly and efficiently remapped from one reference to another similar reference (e.g., between two reference versions or two similar species) using a variety of tools, e.g., the commonly-used CrossMap tool. With the explosion of available genomic data sets and references, high-performance remap** tools will be even more important for kee** up with the computational demands of genome assembly and analysis.
We provide FastRemap, a fast and efficient tool for remap** reads between genome assemblies. FastRemap provides up to a 7.82$\times$ speedup (6.47$\times$, on average) and uses as low as 61.7% (80.7%, on average) of the peak memory consumption compared to the state-of-the-art remap** tool, CrossMap.
FastRemap is written in C++. The source code and user manual are freely available at: github.com/CMU-SAFARI/FastRemap. Docker image available at: https://hub.docker.com/r/alkanlab/fast. Also available in Bioconda at: https://anaconda.org/bioconda/fastremap-bio.
△ Less
Submitted 4 September, 2022; v1 submitted 17 January, 2022;
originally announced January 2022.
-
BLEND: A Fast, Memory-Efficient, and Accurate Mechanism to Find Fuzzy Seed Matches in Genome Analysis
Authors:
Can Firtina,
Jisung Park,
Mohammed Alser,
Jeremie S. Kim,
Damla Senol Cali,
Taha Shahroodi,
Nika Mansouri Ghiasi,
Gagandeep Singh,
Konstantinos Kanellopoulos,
Can Alkan,
Onur Mutlu
Abstract:
Generating the hash values of short subsequences, called seeds, enables quickly identifying similarities between genomic sequences by matching seeds with a single lookup of their hash values. However, these hash values can be used only for finding exact-matching seeds as the conventional hashing methods assign distinct hash values for different seeds, including highly similar seeds. Finding only e…
▽ More
Generating the hash values of short subsequences, called seeds, enables quickly identifying similarities between genomic sequences by matching seeds with a single lookup of their hash values. However, these hash values can be used only for finding exact-matching seeds as the conventional hashing methods assign distinct hash values for different seeds, including highly similar seeds. Finding only exact-matching seeds causes either 1) increasing the use of the costly sequence alignment or 2) limited sensitivity.
We introduce BLEND, the first efficient and accurate mechanism that can identify both exact-matching and highly similar seeds with a single lookup of their hash values, called fuzzy seed matches. BLEND 1) utilizes a technique called SimHash, that can generate the same hash value for similar sets, and 2) provides the proper mechanisms for using seeds as sets with the SimHash technique to find fuzzy seed matches efficiently.
We show the benefits of BLEND when used in read overlap** and read map**. For read overlap**, BLEND is faster by 2.4x - 83.9x (on average 19.3x), has a lower memory footprint by 0.9x - 14.1x (on average 3.8x), and finds higher quality overlaps leading to accurate de novo assemblies than the state-of-the-art tool, minimap2. For read map**, BLEND is faster by 0.8x - 4.1x (on average 1.7x) than minimap2. Source code is available at https://github.com/CMU-SAFARI/BLEND.
△ Less
Submitted 23 May, 2023; v1 submitted 16 December, 2021;
originally announced December 2021.
-
Accelerating Genome Sequence Analysis via Efficient Hardware/Algorithm Co-Design
Authors:
Damla Senol Cali
Abstract:
Genome sequence analysis plays a pivotal role in enabling many medical and scientific advancements in personalized medicine, outbreak tracing, and forensics. However, the analysis of genome sequencing data is currently bottlenecked by the computational power and memory bandwidth limitations of existing systems. In this dissertation, we propose four major works, where we characterize the real-syste…
▽ More
Genome sequence analysis plays a pivotal role in enabling many medical and scientific advancements in personalized medicine, outbreak tracing, and forensics. However, the analysis of genome sequencing data is currently bottlenecked by the computational power and memory bandwidth limitations of existing systems. In this dissertation, we propose four major works, where we characterize the real-system behavior of the genome sequence analysis pipeline and its associated tools, expose the bottlenecks and tradeoffs, and co-design fast and efficient algorithms along with scalable and energy-efficient customized hardware accelerators for the key bottlenecks to enable faster genome sequence analysis.
First, we comprehensively analyze the tools in the genome assembly pipeline for long reads in multiple dimensions, uncovering bottlenecks and tradeoffs that different combinations of tools and different underlying systems lead to. Second, we propose GenASM, an acceleration framework that builds upon bitvector-based approximate string matching to accelerate multiple steps of the genome sequence analysis pipeline. We co-design our highly-parallel, scalable and memory-efficient algorithms with low-power and area-efficient hardware accelerators. Third, we implement an FPGA-based prototype for GenASM, where state-of-the-art 3D-stacked memory offers high memory bandwidth and FPGA resources offer high parallelism. Fourth, we propose SeGraM, the first hardware acceleration framework for sequence-to-graph map** and alignment. We co-design algorithms and accelerators for memory-efficient minimizer-based seeding and bitvector-based, highly-parallel sequence-to-graph alignment.
Overall, we demonstrate that genome sequence analysis can be accelerated by co-designing scalable and energy-efficient customized accelerators along with efficient algorithms for the key steps of genome sequence analysis.
△ Less
Submitted 2 November, 2021;
originally announced November 2021.
-
FPGA-Based Near-Memory Acceleration of Modern Data-Intensive Applications
Authors:
Gagandeep Singh,
Mohammed Alser,
Damla Senol Cali,
Dionysios Diamantopoulos,
Juan Gómez-Luna,
Henk Corporaal,
Onur Mutlu
Abstract:
Modern data-intensive applications demand high computation capabilities with strict power constraints. Unfortunately, such applications suffer from a significant waste of both execution cycles and energy in current computing systems due to the costly data movement between the computation units and the memory units. Genome analysis and weather prediction are two examples of such applications. Recen…
▽ More
Modern data-intensive applications demand high computation capabilities with strict power constraints. Unfortunately, such applications suffer from a significant waste of both execution cycles and energy in current computing systems due to the costly data movement between the computation units and the memory units. Genome analysis and weather prediction are two examples of such applications. Recent FPGAs couple a reconfigurable fabric with high-bandwidth memory (HBM) to enable more efficient data movement and improve overall performance and energy efficiency. This trend is an example of a paradigm shift to near-memory computing. We leverage such an FPGA with high-bandwidth memory (HBM) for improving the pre-alignment filtering step of genome analysis and representative kernels from a weather prediction model. Our evaluation demonstrates large speedups and energy savings over a high-end IBM POWER9 system and a conventional FPGA board with DDR4 memory. We conclude that FPGA-based near-memory computing has the potential to alleviate the data movement bottleneck for modern data-intensive applications.
△ Less
Submitted 3 July, 2021; v1 submitted 11 June, 2021;
originally announced June 2021.
-
GenASM: A High-Performance, Low-Power Approximate String Matching Acceleration Framework for Genome Sequence Analysis
Authors:
Damla Senol Cali,
Gurpreet S. Kalsi,
Zülal Bingöl,
Can Firtina,
Lavanya Subramanian,
Jeremie S. Kim,
Rachata Ausavarungnirun,
Mohammed Alser,
Juan Gomez-Luna,
Amirali Boroumand,
Anant Nori,
Allison Scibisz,
Sreenivas Subramoney,
Can Alkan,
Saugata Ghose,
Onur Mutlu
Abstract:
Genome sequence analysis has enabled significant advancements in medical and scientific areas such as personalized medicine, outbreak tracing, and the understanding of evolution. Unfortunately, it is currently bottlenecked by the computational power and memory bandwidth limitations of existing systems, as many of the steps in genome sequence analysis must process a large amount of data. A major co…
▽ More
Genome sequence analysis has enabled significant advancements in medical and scientific areas such as personalized medicine, outbreak tracing, and the understanding of evolution. Unfortunately, it is currently bottlenecked by the computational power and memory bandwidth limitations of existing systems, as many of the steps in genome sequence analysis must process a large amount of data. A major contributor to this bottleneck is approximate string matching (ASM).
We propose GenASM, the first ASM acceleration framework for genome sequence analysis. We modify the underlying ASM algorithm (Bitap) to significantly increase its parallelism and reduce its memory footprint, and we design the first hardware accelerator for Bitap. Our hardware accelerator consists of specialized compute units and on-chip SRAMs that are designed to match the rate of computation with memory capacity and bandwidth.
We demonstrate that GenASM is a flexible, high-performance, and low-power framework, which provides significant performance and power benefits for three different use cases in genome sequence analysis: 1) GenASM accelerates read alignment for both long reads and short reads. For long reads, GenASM outperforms state-of-the-art software and hardware accelerators by 116x and 3.9x, respectively, while consuming 37x and 2.7x less power. For short reads, GenASM outperforms state-of-the-art software and hardware accelerators by 111x and 1.9x. 2) GenASM accelerates pre-alignment filtering for short reads, with 3.7x the performance of a state-of-the-art pre-alignment filter, while consuming 1.7x less power and significantly improving the filtering accuracy. 3) GenASM accelerates edit distance calculation, with 22-12501x and 9.3-400x speedups over the state-of-the-art software library and FPGA-based accelerator, respectively, while consuming 548-582x and 67x less power.
△ Less
Submitted 16 September, 2020;
originally announced September 2020.
-
Accelerating Genome Analysis: A Primer on an Ongoing Journey
Authors:
Mohammed Alser,
Zülal Bingöl,
Damla Senol Cali,
Jeremie Kim,
Saugata Ghose,
Can Alkan,
Onur Mutlu
Abstract:
Genome analysis fundamentally starts with a process known as read map**, where sequenced fragments of an organism's genome are compared against a reference genome. Read map** is currently a major bottleneck in the entire genome analysis pipeline, because state-of-the-art genome sequencing technologies are able to sequence a genome much faster than the computational techniques employed to analy…
▽ More
Genome analysis fundamentally starts with a process known as read map**, where sequenced fragments of an organism's genome are compared against a reference genome. Read map** is currently a major bottleneck in the entire genome analysis pipeline, because state-of-the-art genome sequencing technologies are able to sequence a genome much faster than the computational techniques employed to analyze the genome. We describe the ongoing journey in significantly improving the performance of read map**. We explain state-of-the-art algorithmic methods and hardware-based acceleration approaches. Algorithmic approaches exploit the structure of the genome as well as the structure of the underlying hardware. Hardware-based acceleration approaches exploit specialized microarchitectures or various execution paradigms (e.g., processing inside or near memory). We conclude with the challenges of adopting these hardware-accelerated read mappers.
△ Less
Submitted 22 September, 2020; v1 submitted 30 July, 2020;
originally announced August 2020.
-
AirLift: A Fast and Comprehensive Technique for Remap** Alignments between Reference Genomes
Authors:
Jeremie S. Kim,
Can Firtina,
Meryem Banu Cavlak,
Damla Senol Cali,
Mohammed Alser,
Nastaran Ha**azar,
Can Alkan,
Onur Mutlu
Abstract:
As genome sequencing tools and techniques improve, researchers are able to incrementally assemble more accurate reference genomes, which enable sensitivity in read map** and downstream analysis such as variant calling. A more sensitive downstream analysis is critical for a better understanding of the genome donor (e.g., health characteristics). Therefore, read sets from sequenced samples should…
▽ More
As genome sequencing tools and techniques improve, researchers are able to incrementally assemble more accurate reference genomes, which enable sensitivity in read map** and downstream analysis such as variant calling. A more sensitive downstream analysis is critical for a better understanding of the genome donor (e.g., health characteristics). Therefore, read sets from sequenced samples should ideally be mapped to the latest available reference genome that represents the most relevant population. Unfortunately, the increasingly large amount of available genomic data makes it prohibitively expensive to fully re-map each read set to its respective reference genome every time the reference is updated. There are several tools that attempt to accelerate the process of updating a read data set from one reference to another (i.e., remap**). However, if a read maps to a region in the old reference that does not appear with a reasonable degree of similarity in the new reference, the read cannot be remapped. We find that, as a result of this drawback, a significant portion of annotations are lost when using state-of-the-art remap** tools. To address this major limitation in existing tools, we propose AirLift, a fast and comprehensive technique for remap** alignments from one genome to another. Compared to the state-of-the-art method for remap** reads (i.e., full map**), AirLift reduces the overall execution time to remap read sets between two reference genome versions by up to 27.4x. We validate our remap** results with GATK and find that AirLift provides high accuracy in identifying ground truth SNP/INDEL variants.
△ Less
Submitted 21 November, 2022; v1 submitted 18 December, 2019;
originally announced December 2019.
-
Understanding the Interactions of Workloads and DRAM Types: A Comprehensive Experimental Study
Authors:
Saugata Ghose,
Tianshi Li,
Nastaran Ha**azar,
Damla Senol Cali,
Onur Mutlu
Abstract:
It has become increasingly difficult to understand the complex interaction between modern applications and main memory, composed of DRAM chips. Manufacturers are now selling and proposing many different types of DRAM, with each DRAM type catering to different needs (e.g., high throughput, low power, high memory density). At the same time, the memory access patterns of prevalent and emerging worklo…
▽ More
It has become increasingly difficult to understand the complex interaction between modern applications and main memory, composed of DRAM chips. Manufacturers are now selling and proposing many different types of DRAM, with each DRAM type catering to different needs (e.g., high throughput, low power, high memory density). At the same time, the memory access patterns of prevalent and emerging workloads are rapidly diverging, as these applications manipulate larger data sets in very different ways. As a result, the combined DRAM-workload behavior is often difficult to intuitively determine today, which can hinder memory optimizations in both hardware and software.
In this work, we identify important families of workloads, as well as prevalent types of DRAM chips, and rigorously analyze the combined DRAM--workload behavior. To this end, we perform a comprehensive experimental study of the interaction between nine different DRAM types and 115 modern applications and multiprogrammed workloads. We draw 12 key observations from our characterization, enabled in part by our development of new metrics that take into account contention between memory requests due to hardware design. Notably, we find that (1) newer DRAM types such as DDR4 and HMC often do not outperform older types such as DDR3, due to higher access latencies and, in the case of HMC, poor exploitation of locality; (2) there is no single DRAM type that can cater to all components of a heterogeneous system (e.g., GDDR5 significantly outperforms other memories for multimedia acceleration, while HMC significantly outperforms other memories for network acceleration); and (3) there is still a strong need to lower DRAM latency, but unfortunately the current design trend of commodity DRAM is toward higher latencies to obtain other benefits. We hope that the trends we identify can drive optimizations in both hardware and software design.
△ Less
Submitted 18 October, 2019; v1 submitted 20 February, 2019;
originally announced February 2019.
-
Apollo: A Sequencing-Technology-Independent, Scalable, and Accurate Assembly Polishing Algorithm
Authors:
Can Firtina,
Jeremie S. Kim,
Mohammed Alser,
Damla Senol Cali,
A. Ercument Cicek,
Can Alkan,
Onur Mutlu
Abstract:
Long reads produced by third-generation sequencing technologies are used to construct an assembly (i.e., the subject's genome), which is further used in downstream genome analysis. Unfortunately, long reads have high sequencing error rates and a large proportion of bps in these long reads are incorrectly identified. These errors propagate to the assembly and affect the accuracy of genome analysis.…
▽ More
Long reads produced by third-generation sequencing technologies are used to construct an assembly (i.e., the subject's genome), which is further used in downstream genome analysis. Unfortunately, long reads have high sequencing error rates and a large proportion of bps in these long reads are incorrectly identified. These errors propagate to the assembly and affect the accuracy of genome analysis. Assembly polishing algorithms minimize such error propagation by polishing or fixing errors in the assembly by using information from alignments between reads and the assembly (i.e., read-to-assembly alignment information). However, assembly polishing algorithms can only polish an assembly using reads either from a certain sequencing technology or from a small assembly. Such technology-dependency and assembly-size dependency require researchers to 1) run multiple polishing algorithms and 2) use small chunks of a large genome to use all available read sets and polish large genomes. We introduce Apollo, a universal assembly polishing algorithm that scales well to polish an assembly of any size (i.e., both large and small genomes) using reads from all sequencing technologies (i.e., second- and third-generation). Our goal is to provide a single algorithm that uses read sets from all available sequencing technologies to improve the accuracy of assembly polishing and that can polish large genomes. Apollo 1) models an assembly as a profile hidden Markov model (pHMM), 2) uses read-to-assembly alignment to train the pHMM with the Forward-Backward algorithm, and 3) decodes the trained model with the Viterbi algorithm to produce a polished assembly. Our experiments with real read sets demonstrate that Apollo is the only algorithm that 1) uses reads from any sequencing technology within a single run and 2) scales well to polish large assemblies without splitting the assembly into multiple parts.
△ Less
Submitted 7 March, 2020; v1 submitted 12 February, 2019;
originally announced February 2019.
-
Nanopore Sequencing Technology and Tools for Genome Assembly: Computational Analysis of the Current State, Bottlenecks and Future Directions
Authors:
Damla Senol Cali,
Jeremie S. Kim,
Saugata Ghose,
Can Alkan,
Onur Mutlu
Abstract:
Nanopore sequencing technology has the potential to render other sequencing technologies obsolete with its ability to generate long reads and provide portability. However, high error rates of the technology pose a challenge while generating accurate genome assemblies. The tools used for nanopore sequence analysis are of critical importance as they should overcome the high error rates of the techno…
▽ More
Nanopore sequencing technology has the potential to render other sequencing technologies obsolete with its ability to generate long reads and provide portability. However, high error rates of the technology pose a challenge while generating accurate genome assemblies. The tools used for nanopore sequence analysis are of critical importance as they should overcome the high error rates of the technology. Our goal in this work is to comprehensively analyze current publicly available tools for nanopore sequence analysis to understand their advantages, disadvantages, and performance bottlenecks. It is important to understand where the current tools do not perform well to develop better tools. To this end, we 1) analyze the multiple steps and the associated tools in the genome assembly pipeline using nanopore sequence data, and 2) provide guidelines for determining the appropriate tools for each step. We analyze various combinations of different tools and expose the tradeoffs between accuracy, performance, memory usage and scalability. We conclude that our observations can guide researchers and practitioners in making conscious and effective choices for each step of the genome assembly pipeline using nanopore sequence data. Also, with the help of bottlenecks we have found, developers can improve the current tools or build new ones that are both accurate and fast, in order to overcome the high error rates of the nanopore sequencing technology.
△ Less
Submitted 5 March, 2018; v1 submitted 23 November, 2017;
originally announced November 2017.
-
GRIM-Filter: Fast Seed Location Filtering in DNA Read Map** Using Processing-in-Memory Technologies
Authors:
Jeremie S. Kim,
Damla Senol Cali,
Hongyi Xin,
Donghyuk Lee,
Saugata Ghose,
Mohammed Alser,
Hasan Hassan,
Oguz Ergin,
Can Alkan,
Onur Mutlu
Abstract:
Motivation: Seed location filtering is critical in DNA read map**, a process where billions of DNA fragments (reads) sampled from a donor are mapped onto a reference genome to identify genomic variants of the donor. State-of-the-art read mappers 1) quickly generate possible map** locations for seeds (i.e., smaller segments) within each read, 2) extract reference sequences at each of the mappin…
▽ More
Motivation: Seed location filtering is critical in DNA read map**, a process where billions of DNA fragments (reads) sampled from a donor are mapped onto a reference genome to identify genomic variants of the donor. State-of-the-art read mappers 1) quickly generate possible map** locations for seeds (i.e., smaller segments) within each read, 2) extract reference sequences at each of the map** locations, and 3) check similarity between each read and its associated reference sequences with a computationally-expensive algorithm (i.e., sequence alignment) to determine the origin of the read. A seed location filter comes into play before alignment, discarding seed locations that alignment would deem a poor match. The ideal seed location filter would discard all poor match locations prior to alignment such that there is no wasted computation on unnecessary alignments.
Results: We propose a novel seed location filtering algorithm, GRIM-Filter, optimized to exploit 3D-stacked memory systems that integrate computation within a logic layer stacked under memory layers, to perform processing-in-memory (PIM). GRIM-Filter quickly filters seed locations by 1) introducing a new representation of coarse-grained segments of the reference genome, and 2) using massively-parallel in-memory operations to identify read presence within each coarse-grained segment. Our evaluations show that for a sequence alignment error tolerance of 0.05, GRIM-Filter 1) reduces the false negative rate of filtering by 5.59x--6.41x, and 2) provides an end-to-end read mapper speedup of 1.81x--3.65x, compared to a state-of-the-art read mapper employing the best previous seed location filtering algorithm.
Availability: The code is available online at: https://github.com/CMU-SAFARI/GRIM
△ Less
Submitted 2 November, 2017;
originally announced November 2017.