A Theory of Multiple-Source Adaptation with Limited Target Labeled Data
Authors:
Yishay Mansour,
Mehryar Mohri,
Jae Ro,
Ananda Theertha Suresh,
Ke Wu
Abstract:
We present a theoretical and algorithmic study of the multiple-source domain adaptation problem in the common scenario where the learner has access only to a limited amount of labeled target data, but where the learner has at disposal a large amount of labeled data from multiple source domains. We show that a new family of algorithms based on model selection ideas benefits from very favorable guar…
▽ More
We present a theoretical and algorithmic study of the multiple-source domain adaptation problem in the common scenario where the learner has access only to a limited amount of labeled target data, but where the learner has at disposal a large amount of labeled data from multiple source domains. We show that a new family of algorithms based on model selection ideas benefits from very favorable guarantees in this scenario and discuss some theoretical obstacles affecting some alternative techniques. We also report the results of several experiments with our algorithms that demonstrate their practical effectiveness.
△ Less
Submitted 29 October, 2020; v1 submitted 19 July, 2020;
originally announced July 2020.
Three Approaches for Personalization with Applications to Federated Learning
Authors:
Yishay Mansour,
Mehryar Mohri,
Jae Ro,
Ananda Theertha Suresh
Abstract:
The standard objective in machine learning is to train a single model for all users. However, in many learning scenarios, such as cloud computing and federated learning, it is possible to learn a personalized model per user. In this work, we present a systematic learning-theoretic study of personalization. We propose and analyze three approaches: user clustering, data interpolation, and model inte…
▽ More
The standard objective in machine learning is to train a single model for all users. However, in many learning scenarios, such as cloud computing and federated learning, it is possible to learn a personalized model per user. In this work, we present a systematic learning-theoretic study of personalization. We propose and analyze three approaches: user clustering, data interpolation, and model interpolation. For all three approaches, we provide learning-theoretic guarantees and efficient algorithms for which we also demonstrate the performance empirically. All of our algorithms are model-agnostic and work for any hypothesis class.
△ Less
Submitted 19 July, 2020; v1 submitted 24 February, 2020;
originally announced February 2020.
