-
Serverless Workflows with Durable Functions and Netherite
Authors:
Sebastian Burckhardt,
Chris Gillum,
David Justo,
Konstantinos Kallas,
Connor McMahon,
Christopher S. Meiklejohn
Abstract:
Serverless is an increasingly popular choice for service architects because it can provide elasticity and load-based billing with minimal developer effort. A common and important use case is to compose serverless functions and cloud storage into reliable workflows. However, existing solutions for authoring workflows provide a rudimentary experience compared to writing standard code in a modern pro…
▽ More
Serverless is an increasingly popular choice for service architects because it can provide elasticity and load-based billing with minimal developer effort. A common and important use case is to compose serverless functions and cloud storage into reliable workflows. However, existing solutions for authoring workflows provide a rudimentary experience compared to writing standard code in a modern programming language. Furthermore, executing workflows reliably in an elastic serverless environment poses significant performance challenges.
To address these, we propose Durable Functions, a programming model for serverless workflows, and Netherite, a distributed execution engine to execute them efficiently. Workflows in Durable Functions are expressed as task-parallel code in a host language of choice. Internally, the workflows are translated to fine-grained stateful communicating processes, which are load-balanced over an elastic cluster. The main challenge is to minimize the cost of reliably persisting progress to storage while supporting elastic scale. Netherite solves this by introducing partitioning, recovery logs, asynchronous snapshots, and speculative communication.
Our results show that Durable Functions simplifies the expression of complex workflows, and that Netherite achieves lower latency and higher throughput than the prevailing approaches for serverless workflows in Azure and AWS, by orders of magnitude in some cases.
△ Less
Submitted 26 February, 2021;
originally announced March 2021.
-
Nota Técnica dos Modelos Implementados pelo Coletivo Covid19br para Projeções de Cenários Futuros da Pandemia COVID-19 no Brasil
Authors:
Daniel Severo,
Giuliano Netto Flores Cruz,
Alcides Carlos de Araújo,
André Marques dos Santos,
André Luiz Nunes Martins,
Carolina Ferreira da Silva,
Cristiane Schmitz,
Felipe Brum de Brito Sousa,
Gabriel Domingos de Arruda,
Gabriel Mendes Cabral Gondim,
Giovanna Ferraresso,
Joao Ricardo Vissoci,
Marcel Figueredo S. Figueredo,
Rafael Prudencio Moreira,
Ralf Lima da Costa,
Vito Ribeiro Venturieri,
Diógenes Adriano Rizzoto Justo
Abstract:
This technical note aims to provide a brief introduction to the projection models used by the group to project future scenarios for states and municipalities in real-time, according to the disease's behavior in previous days. However, the parameters can be modified by the user to design customized scenarios. The proposed model begins with the calculation of the basic reproduction number for the st…
▽ More
This technical note aims to provide a brief introduction to the projection models used by the group to project future scenarios for states and municipalities in real-time, according to the disease's behavior in previous days. However, the parameters can be modified by the user to design customized scenarios. The proposed model begins with the calculation of the basic reproduction number for the state or municipality based on the incidence of cases in the last 12 days. Once this is done, the epidemiological curve is projected using the SEIR compartmentalized epidemic model, in possession of this curve, part of the newly projected infected enter a simulation model for health systems in queuing theory, aiming to project future occupations and collapses.
△ Less
Submitted 25 April, 2020;
originally announced April 2020.
-
Program Synthesis by Type-Guided Abstraction Refinement
Authors:
Zheng Guo,
Michael James,
David Justo,
Jiaxiao Zhou,
Ziteng Wang,
Ranjit Jhala,
Nadia Polikarpova
Abstract:
We consider the problem of type-directed component based synthesis where, given a set of (typed) components and a query type, the goal is to synthesize a term that inhabits the query. Classical approaches based on proof search in intuitionistic logics do not scale up to the standard libraries of modern languages, which span hundreds or thousands of components. Recent graph reachability based metho…
▽ More
We consider the problem of type-directed component based synthesis where, given a set of (typed) components and a query type, the goal is to synthesize a term that inhabits the query. Classical approaches based on proof search in intuitionistic logics do not scale up to the standard libraries of modern languages, which span hundreds or thousands of components. Recent graph reachability based methods proposed for languages like Java do scale, but only apply to components over monomorphic data and functions: polymorphic data and functions infinitely explode the size of the graph that must be searched, rendering synthesis intractable. We introduce type-guided abstraction refinement (TYGAR), a new approach for scalable type-directed synthesis over polymorphic datatypes and components. Our key insight is that we can overcome the explosion by building a graph over abstract types which represent a potentially unbounded set of concrete types. We show how to use graph reachability to search for candidate terms over abstract types, and introduce a new algorithm that uses proofs of untypeability of ill-typed candidates to iteratively refine the abstraction until a well-typed result is found.
We have implemented TYGAR in H+, a tool that takes as input a set of Haskell libraries and a query type, and returns a Haskell term that uses functions from the provided libraries to implement the query type. We have evaluated H+ on a set of 44 queries using a set of popular Haskell libraries with a total of 291 components. Our results demonstrate that H+ returns an interesting solution within the first five results for 33 out of 44 queries. Moreover, TYGAR allows H+ to rapidly return well-typed terms, with the median time to first solution of just 1.4 seconds.
△ Less
Submitted 11 November, 2019;
originally announced November 2019.