Rapid Regression Detection in Software Deployments through Sequential Testing
Authors:
Michael Lindon,
Chris Sanden,
Vaché Shirikian
Abstract:
The practice of continuous deployment has enabled companies to reduce time-to-market by increasing the rate at which software can be deployed. However, deploying more frequently bears the risk that occasionally defective changes are released. For Internet companies, this has the potential to degrade the user experience and increase user abandonment. Therefore, quality control gates are an importan…
▽ More
The practice of continuous deployment has enabled companies to reduce time-to-market by increasing the rate at which software can be deployed. However, deploying more frequently bears the risk that occasionally defective changes are released. For Internet companies, this has the potential to degrade the user experience and increase user abandonment. Therefore, quality control gates are an important component of the software delivery process. These are used to build confidence in the reliability of a release or change. Towards this end, a common approach is to perform a canary test to evaluate new software under production workloads. Detecting defects as early as possible is necessary to reduce exposure and to provide immediate feedback to the developer. We present a statistical framework for rapidly detecting regressions in software deployments. Our approach is based on sequential tests of stochastic order and of equality in distribution. This enables canary tests to be continuously monitored, permitting regressions to be rapidly detected while strictly controlling the false detection probability throughout. The utility of this approach is demonstrated based on two case studies at Netflix.
△ Less
Submitted 22 June, 2022; v1 submitted 29 May, 2022;
originally announced May 2022.
Die Einflüsse von Arbeitsbelastung auf die Arbeitsqualität agiler Software-Entwicklungsteams
Authors:
Christian Sanden,
Kira Karnowski,
Marvin Steinke,
Michael Neumann,
Lukas Linke
Abstract:
Due to the Covid 19 pandemic and the associated effects on the world of work, the burden on employees has been brought into focus. This fact also applies to agile software development teams in many companies due to the extensive switch to remote work. Too high a workload can lead to various negative effects, such as increased sick leave, the well-being of employees, or reduced productivity. It is…
▽ More
Due to the Covid 19 pandemic and the associated effects on the world of work, the burden on employees has been brought into focus. This fact also applies to agile software development teams in many companies due to the extensive switch to remote work. Too high a workload can lead to various negative effects, such as increased sick leave, the well-being of employees, or reduced productivity. It is also known that the workload in knowledge work impacts the quality of the work results. This research article identifies potential factors of the workload of the agile software development team members at Otto GmbH & Co KG. Based on the factors, we present measures to reduce workload and explain our findings, which we have validated in an experiment. Our results show that even small-scale actions, such as the introduction of rest work phases during the working day, lead to positive effects, for example, increased ability to concentrate and how these affect the quality of the work results.
△ Less
Submitted 22 April, 2022;
originally announced April 2022.
