Showing 1–1 of 1 results for author: Bunin, S
-
Conducting A/B Experiments with a Scalable Architecture
Authors:
Andrew Hornback,
Sungeun An,
Scott Bunin,
Stephen Buckley,
John Kos,
Ashok Goel
Abstract:
A/B experiments are commonly used in research to compare the effects of changing one or more variables in two different experimental groups - a control group and a treatment group. While the benefits of using A/B experiments are widely known and accepted, there is less agreement on a principled approach to creating software infrastructure systems to assist in rapidly conducting such experiments. W…
▽ More
A/B experiments are commonly used in research to compare the effects of changing one or more variables in two different experimental groups - a control group and a treatment group. While the benefits of using A/B experiments are widely known and accepted, there is less agreement on a principled approach to creating software infrastructure systems to assist in rapidly conducting such experiments. We propose a four-principle approach for develo** a software architecture to support A/B experiments that is domain agnostic and can help alleviate some of the resource constraints currently needed to successfully implement these experiments: the software architecture (i) must retain the typical properties of A/B experiments, (ii) capture problem solving activities and outcomes, (iii) allow researchers to understand the behavior and outcomes of participants in the experiment, and (iv) must enable automated analysis. We successfully developed a software system to encapsulate these principles and implement it in a real-world A/B experiment.
△ Less
Submitted 23 September, 2023;
originally announced September 2023.