Resonance: Replacing Software Constants with Context-Aware Models in Real-time Communication
Authors:
Jayant Gupchup,
Ashkan Aazami,
Yaran Fan,
Senja Filipi,
Tom Finley,
Scott Inglis,
Marcus Asteborg,
Luke Caroll,
Rajan Chari,
Markus Cozowicz,
Vishak Gopal,
Vinod Prakash,
Sasikanth Bendapudi,
Jack Gerrits,
Eric Lau,
Huazhou Liu,
Marco Rossi,
Dima Slobodianyk,
Dmitri Birjukov,
Matty Cooper,
Nilesh Javar,
Dmitriy Perednya,
Sriram Srinivasan,
John Langford,
Ross Cutler
, et al. (1 additional authors not shown)
Abstract:
Large software systems tune hundreds of 'constants' to optimize their runtime performance. These values are commonly derived through intuition, lab tests, or A/B tests. A 'one-size-fits-all' approach is often sub-optimal as the best value depends on runtime context. In this paper, we provide an experimental approach to replace constants with learned contextual functions for Skype - a widely used r…
▽ More
Large software systems tune hundreds of 'constants' to optimize their runtime performance. These values are commonly derived through intuition, lab tests, or A/B tests. A 'one-size-fits-all' approach is often sub-optimal as the best value depends on runtime context. In this paper, we provide an experimental approach to replace constants with learned contextual functions for Skype - a widely used real-time communication (RTC) application. We present Resonance, a system based on contextual bandits (CB). We describe experiences from three real-world experiments: applying it to the audio, video, and transport components in Skype. We surface a unique and practical challenge of performing machine learning (ML) inference in large software systems written using encapsulation principles. Finally, we open-source FeatureBroker, a library to reduce the friction in adopting ML models in such development environments
△ Less
Submitted 22 November, 2020;
originally announced November 2020.
Making Contextual Decisions with Low Technical Debt
Authors:
Alekh Agarwal,
Sarah Bird,
Markus Cozowicz,
Luong Hoang,
John Langford,
Stephen Lee,
Jiaji Li,
Dan Melamed,
Gal Oshri,
Oswaldo Ribas,
Siddhartha Sen,
Alex Slivkins
Abstract:
Applications and systems are constantly faced with decisions that require picking from a set of actions based on contextual information. Reinforcement-based learning algorithms such as contextual bandits can be very effective in these settings, but applying them in practice is fraught with technical debt, and no general system exists that supports them completely. We address this and create the fi…
▽ More
Applications and systems are constantly faced with decisions that require picking from a set of actions based on contextual information. Reinforcement-based learning algorithms such as contextual bandits can be very effective in these settings, but applying them in practice is fraught with technical debt, and no general system exists that supports them completely. We address this and create the first general system for contextual learning, called the Decision Service.
Existing systems often suffer from technical debt that arises from issues like incorrect data collection and weak debuggability, issues we systematically address through our ML methodology and system abstractions. The Decision Service enables all aspects of contextual bandit learning using four system abstractions which connect together in a loop: explore (the decision space), log, learn, and deploy. Notably, our new explore and log abstractions ensure the system produces correct, unbiased data, which our learner uses for online learning and to enable real-time safeguards, all in a fully reproducible manner.
The Decision Service has a simple user interface and works with a variety of applications: we present two live production deployments for content recommendation that achieved click-through improvements of 25-30%, another with 18% revenue lift in the landing page, and ongoing applications in tech support and machine failure handling. The service makes real-time decisions and learns continuously and scalably, while significantly lowering technical debt.
△ Less
Submitted 9 May, 2017; v1 submitted 13 June, 2016;
originally announced June 2016.
