Showing 1–8 of 8 results for author: Schintke, F

CAWL: A Cache-aware Write Performance Model of Linux Systems

Authors: Masoud Gholami, Florian Schintke

Abstract: The performance of data intensive applications is often dominated by their input/output (I/O) operations but the I/O stack of systems is complex and severely depends on system specific settings and hardware components. This situation makes generic performance optimisation challenging and costly for developers as they would have to run their application on a large variety of systems to evaluate the… ▽ More The performance of data intensive applications is often dominated by their input/output (I/O) operations but the I/O stack of systems is complex and severely depends on system specific settings and hardware components. This situation makes generic performance optimisation challenging and costly for developers as they would have to run their application on a large variety of systems to evaluate their improvements. Here, simulation frameworks can help reducing the experimental overhead but they typically handle the topic of I/O rather coarse-grained, which leads to significant inaccuracies in performance predictions. Here, we propose a more accurate model of the write performance of Linux-based systems that takes different I/O methods and levels (via system calls, library calls, direct or indirect, etc.), the page cache, background writing, and the I/O throttling capabilities of the Linux kernel into account. With our model, we reduce, for example, the relative prediction error compared to a standard I/O model included in SimGrid for a random I/O scenario from 67 % down to 10 % relative error against real measurements of the simulated workload. In other scenarios the differences are even more pronounced. △ Less

Submitted 9 June, 2023; originally announced June 2023.

Comments: 22 pages, 9 figures, 1 table

Validity Constraints for Data Analysis Workflows

Authors: Florian Schintke, Ninon De Mecquenem, David Frantz, Vanessa Emanuela Guarino, Marcus Hilbrich, Fabian Lehmann, Rebecca Sattler, Jan Arne Sparka, Daniel Speckhard, Hermann Stolte, Anh Duc Vu, Ulf Leser

Abstract: Porting a scientific data analysis workflow (DAW) to a cluster infrastructure, a new software stack, or even only a new dataset with some notably different properties is often challenging. Despite the structured definition of the steps (tasks) and their interdependencies during a complex data analysis in the DAW specification, relevant assumptions may remain unspecified and implicit. Such hidden a… ▽ More Porting a scientific data analysis workflow (DAW) to a cluster infrastructure, a new software stack, or even only a new dataset with some notably different properties is often challenging. Despite the structured definition of the steps (tasks) and their interdependencies during a complex data analysis in the DAW specification, relevant assumptions may remain unspecified and implicit. Such hidden assumptions often lead to crashing tasks without a reasonable error message, poor performance in general, non-terminating executions, or silent wrong results of the DAW, to name only a few possible consequences. Searching for the causes of such errors and drawbacks in a distributed compute cluster managed by a complex infrastructure stack, where DAWs for large datasets typically are executed, can be tedious and time-consuming. We propose validity constraints (VCs) as a new concept for DAW languages to alleviate this situation. A VC is a constraint specifying some logical conditions that must be fulfilled at certain times for DAW executions to be valid. When defined together with a DAW, VCs help to improve the portability, adaptability, and reusability of DAWs by making implicit assumptions explicit. Once specified, VC can be controlled automatically by the DAW infrastructure, and violations can lead to meaningful error messages and graceful behaviour (e.g., termination or invocation of repair mechanisms). We provide a broad list of possible VCs, classify them along multiple dimensions, and compare them to similar concepts one can find in related fields. We also provide a first sketch for VCs' implementation into existing DAW infrastructures. △ Less

Submitted 15 May, 2023; originally announced May 2023.

BottleMod: Modeling Data Flows and Tasks for Fast Bottleneck Analysis

Authors: Ansgar Lößer, Joel Witzke, Florian Schintke, Björn Scheuermann

Abstract: In the recent years, scientific workflows gained more and more popularity. In scientific workflows, tasks are typically treated as black boxes. Dealing with their complex interrelations to identify optimization potentials and bottlenecks is therefore inherently hard. The progress of a scientific workflow depends on several factors, including the available input data, the available computational po… ▽ More In the recent years, scientific workflows gained more and more popularity. In scientific workflows, tasks are typically treated as black boxes. Dealing with their complex interrelations to identify optimization potentials and bottlenecks is therefore inherently hard. The progress of a scientific workflow depends on several factors, including the available input data, the available computational power, and the I/O and network bandwidth. Here, we tackle the problem of predicting the workflow progress with very low overhead. To this end, we look at suitable formalizations for the key parameters and their interactions which are sufficiently flexible to describe the input data consumption, the computational effort and the output production of the workflow's tasks. At the same time they allow for computationally simple and fast performance predictions, including a bottleneck analysis over the workflow runtime. A piecewise-defined bottleneck function is derived from the discrete intersections of the task models' limiting functions. This allows to estimate potential performance gains from overcoming the bottlenecks and can be used as a basis for optimized resource allocation and workflow execution. △ Less

Submitted 12 September, 2022; originally announced September 2022.

Transactions on Red-black and AVL trees in NVRAM

Authors: Thorsten Schütt, Florian Schintke, Jan Skrzypczak

Abstract: Byte-addressable non-volatile memory (NVRAM) supports persistent storage with low latency and high bandwidth. Complex data structures in it ought to be updated transactionally, so that they remain recoverable at all times. Traditional database technologies such as kee** a separate log, a journal, or shadow data work on a coarse-grained level, where the whole transaction is made visible using a f… ▽ More Byte-addressable non-volatile memory (NVRAM) supports persistent storage with low latency and high bandwidth. Complex data structures in it ought to be updated transactionally, so that they remain recoverable at all times. Traditional database technologies such as kee** a separate log, a journal, or shadow data work on a coarse-grained level, where the whole transaction is made visible using a final atomic update operation. These methods typically need significant additional space overhead and induce non-trivial overhead for log pruning, state maintenance, and resource (de-)allocation. Thus, they are not necessarily the best choice for NVRAM, which supports fine-grained, byte-addressable access. We present a generic transaction mechanism to update dynamic complex data structures `in-place' with a constant memory overhead. It is independent of the size of the data structure. We demonstrate and evaluate our approach on Red-Black Trees and AVL Trees with a redo log of constant size (4 resp. 2 cache lines). The redo log guarantees that each accepted (started) transaction is executed eventually despite arbitrary many system crashes and recoveries in the meantime. We update complex data structures in local and remote NVRAM providing exactly once semantics and durable linearizability for multi-reader single-writer access. To persist data, we use the available processor instructions for NVRAM in the local case and remote direct memory access (RDMA) combined with a software agent in the remote case. △ Less

Submitted 29 June, 2020; originally announced June 2020.

RMWPaxos: Fault-Tolerant In-Place Consensus Sequences

Authors: Jan Skrzypczak, Florian Schintke, Thorsten Schütt

Abstract: Building consensus sequences based on distributed, fault-tolerant consensus, as used for replicated state machines, typically requires a separate distributed state for every new consensus instance. Allocating and maintaining this state causes significant overhead. In particular, freeing the distributed, outdated states in a fault-tolerant way is not trivial and adds further complexity and cost to… ▽ More Building consensus sequences based on distributed, fault-tolerant consensus, as used for replicated state machines, typically requires a separate distributed state for every new consensus instance. Allocating and maintaining this state causes significant overhead. In particular, freeing the distributed, outdated states in a fault-tolerant way is not trivial and adds further complexity and cost to the system. In this paper, we propose an extension to the single-decree Paxos protocol that can learn a sequence of consensus decisions 'in-place', i.e. with a single set of distributed states. Our protocol does not require dynamic log structures and hence has no need for distributed log pruning, snapshotting, compaction, or dynamic resource allocation. The protocol builds a fault-tolerant atomic register that supports arbitrary read-modify-write operations. We use the concept of consistent quorums to detect whether the previous consensus still needs to be consolidated or is already finished so that the next consensus value can be safely proposed. Reading a consolidated consensus is done without state modifications and is thereby free of concurrency control and demand for serialisation. A proposer that is not interrupted reaches agreement on consecutive consensus decisions within a single message round-trip per decision by preparing the acceptors eagerly with the previous request. △ Less

Submitted 1 April, 2020; v1 submitted 10 January, 2020; originally announced January 2020.

Learned Clause Minimization in Parallel SAT Solvers

Authors: Marc Hartung, Florian Schintke

Abstract: Learned clauses minimization (LCM) let to performance improvements of modern SAT solvers especially in solving hard SAT instances. Despite the success of LCM approaches in sequential solvers, they are not widely incorporated in parallel SAT solvers. In this paper we explore the potential of LCM for parallel SAT solvers by defining multiple LCM approaches based on clause vivification, comparing the… ▽ More Learned clauses minimization (LCM) let to performance improvements of modern SAT solvers especially in solving hard SAT instances. Despite the success of LCM approaches in sequential solvers, they are not widely incorporated in parallel SAT solvers. In this paper we explore the potential of LCM for parallel SAT solvers by defining multiple LCM approaches based on clause vivification, comparing their runtime in different SAT solvers and discussing reasons for performance gains and losses. Results show that LCM only boosts performance of parallel SAT solvers on a fraction of SAT instances. More commonly applying LCM decreases performance. Only certain LCM approaches are able to improve the overall performance of parallel SAT solvers. △ Less

Submitted 5 August, 2019; originally announced August 2019.

Comments: accepted at Pragmatics of SAT 2019

Linearizable State Machine Replication of State-Based CRDTs without Logs

Authors: Jan Skrzypczak, Florian Schintke, Thorsten Schütt

Abstract: General solutions of state machine replication have to ensure that all replicas apply the same commands in the same order, even in the presence of failures. Such strict ordering incurs high synchronization costs caused by distributed consensus or by the use of a leader. This paper presents a protocol for linearizable state machine replication of conflict-free replicated data types (CRDTs) that n… ▽ More General solutions of state machine replication have to ensure that all replicas apply the same commands in the same order, even in the presence of failures. Such strict ordering incurs high synchronization costs caused by distributed consensus or by the use of a leader. This paper presents a protocol for linearizable state machine replication of conflict-free replicated data types (CRDTs) that neither requires consensus nor a leader. By leveraging the properties of state-based CRDTs - in particular, the monotonic growth of a join semilattice - synchronization overhead is greatly reduced. As a result, updates only need a single round trip and modify the state 'in-place' without the need for a log. Furthermore, the message size overhead for coordination consists of a single counter per message. For queries, we guarantee finite writes termination. We show in an experimental evaluation that more than 99 % of queries can be handled in one to three round trips under highly concurrent accesses. Our protocol achieves high throughput without auxiliary processes such as command log management or leader election. Thus, it is well suited for practical scenarios that need linearizable access to CRDT data on a fine-granular scale. △ Less

Submitted 24 July, 2020; v1 submitted 21 May, 2019; originally announced May 2019.

Challenges in QCD matter physics - The Compressed Baryonic Matter experiment at FAIR

Authors: CBM Collaboration, T. Ablyazimov, A. Abuhoza, R. P. Adak, M. Adamczyk, K. Agarwal, M. M. Aggarwal, Z. Ahammed, F. Ahmad, N. Ahmad, S. Ahmad, A. Akindinov, P. Akishin, E. Akishina, T. Akishina, V. Akishina, A. Akram, M. Al-Turany, I. Alekseev, E. Alexandrov, I. Alexandrov, S. Amar-Youcef, M. Anđelić, O. Andreeva, C. Andrei , et al. (563 additional authors not shown)

Abstract: Substantial experimental and theoretical efforts worldwide are devoted to explore the phase diagram of strongly interacting matter. At LHC and top RHIC energies, QCD matter is studied at very high temperatures and nearly vanishing net-baryon densities. There is evidence that a Quark-Gluon-Plasma (QGP) was created at experiments at RHIC and LHC. The transition from the QGP back to the hadron gas is… ▽ More Substantial experimental and theoretical efforts worldwide are devoted to explore the phase diagram of strongly interacting matter. At LHC and top RHIC energies, QCD matter is studied at very high temperatures and nearly vanishing net-baryon densities. There is evidence that a Quark-Gluon-Plasma (QGP) was created at experiments at RHIC and LHC. The transition from the QGP back to the hadron gas is found to be a smooth cross over. For larger net-baryon densities and lower temperatures, it is expected that the QCD phase diagram exhibits a rich structure, such as a first-order phase transition between hadronic and partonic matter which terminates in a critical point, or exotic phases like quarkyonic matter. The discovery of these landmarks would be a breakthrough in our understanding of the strong interaction and is therefore in the focus of various high-energy heavy-ion research programs. The Compressed Baryonic Matter (CBM) experiment at FAIR will play a unique role in the exploration of the QCD phase diagram in the region of high net-baryon densities, because it is designed to run at unprecedented interaction rates. High-rate operation is the key prerequisite for high-precision measurements of multi-differential observables and of rare diagnostic probes which are sensitive to the dense phase of the nuclear fireball. The goal of the CBM experiment at SIS100 (sqrt(s_NN) = 2.7 - 4.9 GeV) is to discover fundamental properties of QCD matter: the phase structure at large baryon-chemical potentials (mu_B > 500 MeV), effects of chiral symmetry, and the equation-of-state at high density as it is expected to occur in the core of neutron stars. In this article, we review the motivation for and the physics programme of CBM, including activities before the start of data taking in 2022, in the context of the worldwide efforts to explore high-density QCD matter. △ Less

Submitted 29 March, 2017; v1 submitted 6 July, 2016; originally announced July 2016.

Comments: 15 pages, 11 figures. Published in European Physical Journal A

Journal ref: Eur. Phys. J. A 53 (2017) 60