Showing 1–1 of 1 results for author: Akoush, S

Desiderata for next generation of ML model serving

Authors: Sherif Akoush, Andrei Paleyes, Arnaud Van Looveren, Clive Cox

Abstract: Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in p… ▽ More Inference is a significant part of ML software infrastructure. Despite the variety of inference frameworks available, the field as a whole can be considered in its early days. This position paper puts forth a range of important qualities that next generation of inference platforms should be aiming for. We present our rationale for the importance of each quality, and discuss ways to achieve it in practice. We propose to focus on data-centricity as the overarching design pattern which enables smarter ML system deployment and operation at scale. △ Less

Submitted 22 November, 2022; v1 submitted 26 October, 2022; originally announced October 2022.

Comments: Accepted at NeurIPS 2022 Workshop on Challenges in Deploying and Monitoring Machine Learning Systems