-
Diba: A Re-configurable Stream Processor
Authors:
Mohammadreza Najafi,
Thamir M. Qadah,
Mohammad Sadoghi,
Hans-Arno Jacobsen
Abstract:
Stream processing acceleration is driven by the continuously increasing volume and velocity of data generated on the Web and the limitations of storage, computation, and power consumption. Hardware solutions provide better performance and power consumption, but they are hindered by the high research and development costs and the long time to market. In this work, we propose our re-configurable str…
▽ More
Stream processing acceleration is driven by the continuously increasing volume and velocity of data generated on the Web and the limitations of storage, computation, and power consumption. Hardware solutions provide better performance and power consumption, but they are hindered by the high research and development costs and the long time to market. In this work, we propose our re-configurable stream processor (Diba), a complete rethinking of a previously proposed customized and flexible query processor that targets real-time stream processing. Diba uses a unidirectional dataflow not dedicated to any specific type of query (operator) on streams, allowing a straightforward placement of processing components on a general data path that facilitates query map**. In Diba, the concepts of the distribution network and processing components are implemented as two separate entities connected using generic interfaces. This approach allows the adoption of a versatile architecture for a family of queries rather than forcing a rigid chain of processing components to implement such queries. Our experimental evaluations of representative queries from TPC-H yielded processing times of 300, 1220, and 3520 milliseconds for data streams with scale factor sizes of one, four, and ten gigabytes, respectively.
△ Less
Submitted 4 April, 2023;
originally announced April 2023.
-
Highly Available Queue-oriented Speculative Transaction Processing
Authors:
Thamir M. Qadah,
Mohammad Sadoghi
Abstract:
Deterministic database systems have received increasing attention from the database research community in recent years. Despite their current limitations, recent proposals of distributed deterministic transaction processing systems demonstrated significant improvements over systems using traditional transaction processing techniques (e.g., two-phase-locking or optimistic concurrency control with t…
▽ More
Deterministic database systems have received increasing attention from the database research community in recent years. Despite their current limitations, recent proposals of distributed deterministic transaction processing systems demonstrated significant improvements over systems using traditional transaction processing techniques (e.g., two-phase-locking or optimistic concurrency control with two-phase-commit). However, the problem of ensuring high availability in deterministic distributed transaction processing systems has received less attention from the research community, and this aspect has not been analyzed and evaluated well. This paper proposes a generic framework to model the replication process in deterministic transaction processing systems and use it to study three cases. We design and implement QR-Store, a queue-oriented replicated transaction processing system, and extensively evaluate it with various workloads based on a transactional version of YCSB. Our prototype implementation QR-Store can achieve a throughput of 1.9 million replicated transactions per second in under 200 milliseconds and a replication overhead of 8%-25% compared to non-replicated configurations.
△ Less
Submitted 23 July, 2021;
originally announced July 2021.
-
A Queue-oriented Transaction Processing Paradigm
Authors:
Thamir M. Qadah
Abstract:
Transaction processing has been an active area of research for several decades. A fundamental characteristic of classical transaction processing protocols is non-determinism, which causes them to suffer from performance issues on modern computing environments such as main-memory databases using many-core, and multi-socket CPUs and distributed environments. Recent proposals of deterministic transac…
▽ More
Transaction processing has been an active area of research for several decades. A fundamental characteristic of classical transaction processing protocols is non-determinism, which causes them to suffer from performance issues on modern computing environments such as main-memory databases using many-core, and multi-socket CPUs and distributed environments. Recent proposals of deterministic transaction processing techniques have shown great potential in addressing these performance issues. In this position paper, I argue for a queue-oriented transaction processing paradigm that leads to better design and implementation of deterministic transaction processing protocols. I support my approach with extensive experimental evaluations and demonstrate significant performance gains.
△ Less
Submitted 23 October, 2019;
originally announced October 2019.
