Computer Science > Machine Learning
[Submitted on 27 Feb 2021 (v1), revised 11 Aug 2021 (this version, v4), latest version 29 Jan 2022 (v6)]
Title:Neural Architecture Search From Task Similarity Measure
View PDFAbstract:In this paper, we propose a neural architecture search framework based on a similarity measure between the baseline tasks and the incoming target task. We first define the notion of task similarity based on the log-determinant of the Fisher Information Matrices. Next, we compute the task similarity from each of the baseline tasks to the incoming target task. By utilizing the relation between a target and a set of learned baseline tasks, the search space of architectures for the incoming target task can be significantly reduced, making the discovery of the best candidates in the set of possible architectures tractable and efficient, in terms of GPU days. This method eliminates the requirement for training the networks from scratch for the incoming target task as well as introducing the bias in the initialization of the search space from the human domain. Experimental results with 8 classification tasks in MNIST and CIFAR-10 datasets illustrate the efficacy of our proposed approach and its competitiveness with other state-of-art methods in terms of the classification performance, the number of parameters, and the search time.
Submission history
From: Cat Le [view email][v1] Sat, 27 Feb 2021 15:26:14 UTC (63 KB)
[v2] Wed, 3 Mar 2021 14:53:53 UTC (63 KB)
[v3] Mon, 15 Mar 2021 22:05:22 UTC (63 KB)
[v4] Wed, 11 Aug 2021 23:09:01 UTC (19,948 KB)
[v5] Thu, 7 Oct 2021 00:17:46 UTC (73 KB)
[v6] Sat, 29 Jan 2022 16:06:34 UTC (73 KB)
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
Connected Papers (What is Connected Papers?)
CORE Recommender (What is CORE?)
IArxiv Recommender
(What is IArxiv?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.