-
Logical Synchrony Networks: A formal model for deterministic distribution
Authors:
Logan Kenwright,
Partha Roop,
Nathan Allen,
Sanjay Lall,
Calin Cascaval,
Tammo Spalink,
Martin Izzard
Abstract:
Kahn Process Networks (KPNs) are a deterministic Model of Computation (MoC) for distributed systems. KPNs supports non-blocking writes and blocking reads, with the consequent assumption of unbounded buffers between processes. Variants such as Finite FIFO Platforms (FFP) have been developed, which enforce boundedness. One issue with existing models is that they mix process synchronisation with proc…
▽ More
Kahn Process Networks (KPNs) are a deterministic Model of Computation (MoC) for distributed systems. KPNs supports non-blocking writes and blocking reads, with the consequent assumption of unbounded buffers between processes. Variants such as Finite FIFO Platforms (FFP) have been developed, which enforce boundedness. One issue with existing models is that they mix process synchronisation with process execution. In this paper we address how these two facets may be decoupled.
This paper explores a recent alternative called bittide, which decouples the execution of a process from the control needed for process synchronisation, and thus preserves determinism and boundedness while ensuring pipelined execution for better throughput. Our intuition is that such an approach could leverage not only determinism and buffer boundedness but may potentially offer better overall throughput.
To understand the behavior of these systems we define a formal model -- a deterministic MoC called Logical Synchrony Networks (LSNs). LSNs describes a network of processes modelled as a graph, with edges representing invariant logical delays between a producer process and the corresponding consumer process. We show that this abstraction is satisfied by KPNs. Subsequently, we show that both FFPs and bittide faithfully implement this abstraction. Thus, we show for the first time that FFPs and bittide offer two alternative ways of implementing deterministic distributed systems with the latter being more performant.
△ Less
Submitted 5 June, 2024; v1 submitted 12 February, 2024;
originally announced February 2024.
-
Logical Synchrony and the bittide Mechanism
Authors:
Sanjay Lall,
Calin Cascaval,
Martin Izzard,
Tammo Spalink
Abstract:
We introduce logical synchrony, a framework that allows distributed computing to be coordinated as tightly as in synchronous systems without the distribution of a global clock or any reference to universal time. We develop a model of events called a logical synchrony network, in which nodes correspond to processors and every node has an associated local clock which generates the events. We constru…
▽ More
We introduce logical synchrony, a framework that allows distributed computing to be coordinated as tightly as in synchronous systems without the distribution of a global clock or any reference to universal time. We develop a model of events called a logical synchrony network, in which nodes correspond to processors and every node has an associated local clock which generates the events. We construct a measure of logical latency and develop its properties. A further model, called a multiclock network, is then analyzed and shown to be a refinement of the logical synchrony network. We present the bittide mechanism as an instantiation of multiclock networks, and discuss the clock control mechanism that ensures that buffers do not overflow or underflow. Finally we give conditions under which a logical synchrony network has an equivalent synchronous realization.
△ Less
Submitted 8 May, 2024; v1 submitted 31 July, 2023;
originally announced August 2023.
-
On Buffer Centering for Bittide Synchronization
Authors:
Sanjay Lall,
Calin Cascaval,
Martin Izzard,
Tammo Spalink
Abstract:
We discuss distributed reframing control of bittide systems. In a bittide system, multiple processors synchronize by monitoring communication over the network. The processors remain in logical synchrony by controlling the timing of frame transmissions. The protocol for doing this relies upon an underlying dynamic control system, where each node makes only local observations and performs no direct…
▽ More
We discuss distributed reframing control of bittide systems. In a bittide system, multiple processors synchronize by monitoring communication over the network. The processors remain in logical synchrony by controlling the timing of frame transmissions. The protocol for doing this relies upon an underlying dynamic control system, where each node makes only local observations and performs no direct coordination with other nodes. In this paper we develop a control algorithm based on the idea of reset control, which allows all nodes to maintain small buffer offsets while also requiring very little state information at each node. We demonstrate that with reframing, we can achieve separate control of frequency and phase, allowing both the frequencies to be syntonized and the buffers to be moved the desired points, rather than combining their control via a proportional-integral controller. This offers the potential for simplified boot processes and failure handling.
△ Less
Submitted 20 March, 2023;
originally announced March 2023.
-
Resistance Distance and Control Performance for bittide Synchronization
Authors:
Sanjay Lall,
Calin Cascaval,
Martin Izzard,
Tammo Spalink
Abstract:
We discuss control of bittide distributed systems, which are designed to provide logical synchronization between networked machines by observing data flow rates between adjacent systems at the physical network layer and controlling local reference clock frequencies. We analyze the performance of approximate proportional-integral control of the synchronization mechanism and develop a simple continu…
▽ More
We discuss control of bittide distributed systems, which are designed to provide logical synchronization between networked machines by observing data flow rates between adjacent systems at the physical network layer and controlling local reference clock frequencies. We analyze the performance of approximate proportional-integral control of the synchronization mechanism and develop a simple continuous-time model to show the resulting dynamics are stable for any positive choice of gains. We then construct explicit formulae to show that closed-loop performance measured using the L2 norm is a product of two terms, one depending only on resistance distances in the graph, and the other depending only on controller gains.
△ Less
Submitted 31 March, 2022; v1 submitted 9 November, 2021;
originally announced November 2021.
-
Modeling and Control of bittide Synchronization
Authors:
Sanjay Lall,
Calin Cascaval,
Martin Izzard,
Tammo Spalink
Abstract:
Distributed system applications rely on a fine-grain common sense of time. Existing systems maintain the common sense of time by kee** each independent machine as close as possible to wall-clock time through a combination of software protocols like NTP and GPS signals and/or precision references like atomic clocks. This approach is expensive and has tolerance limitations that require protocols t…
▽ More
Distributed system applications rely on a fine-grain common sense of time. Existing systems maintain the common sense of time by kee** each independent machine as close as possible to wall-clock time through a combination of software protocols like NTP and GPS signals and/or precision references like atomic clocks. This approach is expensive and has tolerance limitations that require protocols to deal with asynchrony and its performance consequences. Moreover, at data-center scale it is impractical to distribute a physical clock as is done on a chip or printed circuit board. In this paper we introduce a distributed system design that removes the need for physical clock distribution or mechanisms for maintaining close alignment to wall-clock time, and instead provides applications with a perfectly synchronized logical clock. We discuss the abstract frame model (AFM), a mathematical model that underpins the system synchronization. The model is based on the rate of communication between nodes in a topology without requiring a global clock. We show that there are families of controllers that satisfy the properties required for existence and uniqueness of solutions to the AFM, and give examples.
△ Less
Submitted 31 March, 2022; v1 submitted 28 September, 2021;
originally announced September 2021.
-
Programming Protocol-Independent Packet Processors
Authors:
Pat Bosshart,
Dan Daly,
Martin Izzard,
Nick McKeown,
Jennifer Rexford,
Cole Schlesinger,
Dan Talayco,
Amin Vahdat,
George Varghese,
David Walker
Abstract:
P4 is a high-level language for programming protocol-independent packet processors. P4 works in conjunction with SDN control protocols like OpenFlow. In its current form, OpenFlow explicitly specifies protocol headers on which it operates. This set has grown from 12 to 41 fields in a few years, increasing the complexity of the specification while still not providing the flexibility to add new head…
▽ More
P4 is a high-level language for programming protocol-independent packet processors. P4 works in conjunction with SDN control protocols like OpenFlow. In its current form, OpenFlow explicitly specifies protocol headers on which it operates. This set has grown from 12 to 41 fields in a few years, increasing the complexity of the specification while still not providing the flexibility to add new headers. In this paper we propose P4 as a strawman proposal for how OpenFlow should evolve in the future. We have three goals: (1) Reconfigurability in the field: Programmers should be able to change the way switches process packets once they are deployed. (2) Protocol independence: Switches should not be tied to any specific network protocols. (3) Target independence: Programmers should be able to describe packet-processing functionality independently of the specifics of the underlying hardware. As an example, we describe how to use P4 to configure a switch to add a new hierarchical label.
△ Less
Submitted 15 May, 2014; v1 submitted 5 December, 2013;
originally announced December 2013.
-
Sintering behavior of two roughened crystals just after contact
Authors:
R. S. Farr,
M. J. Izzard
Abstract:
We consider two spherical, roughened crystals with approximately isotropic surface free energy which are brought into contact and begin to sinter. We argue that the geometry immediately post-contact is two dimensional and Cartesian and can be approximated by the evolution of a slot-shaped cavity. On this basis, we construct travelling wave solutions for the crystal shape in the limits of bulk di…
▽ More
We consider two spherical, roughened crystals with approximately isotropic surface free energy which are brought into contact and begin to sinter. We argue that the geometry immediately post-contact is two dimensional and Cartesian and can be approximated by the evolution of a slot-shaped cavity. On this basis, we construct travelling wave solutions for the crystal shape in the limits of bulk diffusion limited and surface diffusion limited kinetics. These solutions are then used to calculate novel scalings for the neck size as a function of time $t$ after contact: We predict that neck size is proportional to $t^{1/4}$ for the bulk diffusion limited case and (following a single pinch-off event) approximately proportional to $t^{1/3}$ for the surface diffusion limited case.
△ Less
Submitted 22 January, 2010;
originally announced January 2010.
-
The Tiny Tera: A Packet Switch Core
Authors:
Nick McKeown,
Martin Izzard,
Adisak Mekkittikul,
Bill Ellersick,
Mark Horowitz
Abstract:
The objective is to design and build a small, high-bandwidth switch.
The objective is to design and build a small, high-bandwidth switch.
△ Less
Submitted 5 October, 1998;
originally announced October 1998.
