-
NAS-HPO-Bench-II: A Benchmark Dataset on Joint Optimization of Convolutional Neural Network Architecture and Training Hyperparameters
Authors:
Yoichi Hirose,
Nozomu Yoshinari,
Shinichi Shirakawa
Abstract:
The benchmark datasets for neural architecture search (NAS) have been developed to alleviate the computationally expensive evaluation process and ensure a fair comparison. Recent NAS benchmarks only focus on architecture optimization, although the training hyperparameters affect the obtained model performances. Building the benchmark dataset for joint optimization of architecture and training hype…
▽ More
The benchmark datasets for neural architecture search (NAS) have been developed to alleviate the computationally expensive evaluation process and ensure a fair comparison. Recent NAS benchmarks only focus on architecture optimization, although the training hyperparameters affect the obtained model performances. Building the benchmark dataset for joint optimization of architecture and training hyperparameters is essential to further NAS research. The existing NAS-HPO-Bench is a benchmark for joint optimization, but it does not consider the network connectivity design as done in modern NAS algorithms. This paper introduces the first benchmark dataset for joint optimization of network connections and training hyperparameters, which we call NAS-HPO-Bench-II. We collect the performance data of 4K cell-based convolutional neural network architectures trained on the CIFAR-10 dataset with different learning rate and batch size settings, resulting in the data of 192K configurations. The dataset includes the exact data for 12 epoch training. We further build the surrogate model predicting the accuracies after 200 epoch training to provide the performance data of longer training epoch. By analyzing NAS-HPO-Bench-II, we confirm the dependency between architecture and training hyperparameters and the necessity of joint optimization. Finally, we demonstrate the benchmarking of the baseline optimization algorithms using NAS-HPO-Bench-II.
△ Less
Submitted 19 October, 2021;
originally announced October 2021.
-
The Prominent Charge-Transfer Effects of Trinuclear Complexes with Nominally High Nickel Valences
Authors:
K. Yamagami,
S. Imada,
K. Yamanaka,
T. Yaji,
A. Tanaka,
M. Kouno,
N. Yoshinari,
T. Konno,
A. Sekiyama
Abstract:
Recently synthesized Rh-Ni trinuclear complexes hexacoordinated with sulfur ions, 3-aminopropanethiolate (apt) metalloligand [Ni{Rh(apt)$_{3}$}$_{2}$](NO$_{3}$)$_{n}$ ($n$ = 2, 3, 4), are found to be chemically interconvertible between the nominal Ni$^{2+}$ and Ni$^{4+}$ states. In order to clarify the origins of their interconvertible nature and the stability of such a high oxidation state as the…
▽ More
Recently synthesized Rh-Ni trinuclear complexes hexacoordinated with sulfur ions, 3-aminopropanethiolate (apt) metalloligand [Ni{Rh(apt)$_{3}$}$_{2}$](NO$_{3}$)$_{n}$ ($n$ = 2, 3, 4), are found to be chemically interconvertible between the nominal Ni$^{2+}$ and Ni$^{4+}$ states. In order to clarify the origins of their interconvertible nature and the stability of such a high oxidation state as the tetravalency from the physical point of view, we have systematically investigated the local 3$d$ electronic structures of [Ni{Rh(apt)$_{3}$}$_{2}$](NO$_{3}$)$_{n}$ by means of soft X-ray core-level absorption spectroscopy (XAS). The experimental data have been reproduced by the single-site configuration-interaction cluster-model simulations, which indicate that the charge-transferred configurations are more stable than the nominal $d$-electron-number configuration for $n=3,4$ leading to the prominent charge-transfer effects. These are also supported by S $K$-edge XAS of [Ni{Rh(apt)$_{3}$}$_{2}$](NO$_{3}$)$_{n}$. Our results imply that the found charge-transfer effects have a key role to realize the interconvertible nature as well as the stability of the high oxidization state of the Ni ions.
△ Less
Submitted 28 April, 2020;
originally announced April 2020.
-
Adaptive Stochastic Natural Gradient Method for One-Shot Neural Architecture Search
Authors:
Youhei Akimoto,
Shinichi Shirakawa,
Nozomu Yoshinari,
Kento Uchida,
Shota Saito,
Kouhei Nishida
Abstract:
High sensitivity of neural architecture search (NAS) methods against their input such as step-size (i.e., learning rate) and search space prevents practitioners from applying them out-of-the-box to their own problems, albeit its purpose is to automate a part of tuning process. Aiming at a fast, robust, and widely-applicable NAS, we develop a generic optimization framework for NAS. We turn a couple…
▽ More
High sensitivity of neural architecture search (NAS) methods against their input such as step-size (i.e., learning rate) and search space prevents practitioners from applying them out-of-the-box to their own problems, albeit its purpose is to automate a part of tuning process. Aiming at a fast, robust, and widely-applicable NAS, we develop a generic optimization framework for NAS. We turn a coupled optimization of connection weights and neural architecture into a differentiable optimization by means of stochastic relaxation. It accepts arbitrary search space (widely-applicable) and enables to employ a gradient-based simultaneous optimization of weights and architecture (fast). We propose a stochastic natural gradient method with an adaptive step-size mechanism built upon our theoretical investigation (robust). Despite its simplicity and no problem-dependent parameter tuning, our method exhibited near state-of-the-art performances with low computational budgets both on image classification and inpainting tasks.
△ Less
Submitted 21 May, 2019;
originally announced May 2019.
-
Local 3d Electronic Structures of Co-Based Complexes with Medicinal Molecules Probed by Soft X-Ray Absorption
Authors:
Kohei Yamagami,
Hidenori Fujiwara,
Shin Imada,
Toshiharu Kadono,
Keisuke Yamanaka,
Takayuki Muro,
Arata Tanaka,
Takuma Itai,
Nobuto Yoshinari,
Takumi Konno,
Akira Sekiyama
Abstract:
We have examined the local 3d electronic structures of Co-Au multinuclear complexes with the medicinal molecules D-penicillaminate (D-pen) [Co{Au(PPh3)(D-pen)}2]ClO4 and [Co3{Au3(tdme)(D-pen)3}2] by Co L_2,3-edge soft X-ray absorption (XAS) spectroscopy, where PPh3 denotes triphenylphosphine and tdme stands for 1,1,1-tris[(diphenylphosphino)methyl]ethane. The Co L_2,3-edge XAS spectra indicate the…
▽ More
We have examined the local 3d electronic structures of Co-Au multinuclear complexes with the medicinal molecules D-penicillaminate (D-pen) [Co{Au(PPh3)(D-pen)}2]ClO4 and [Co3{Au3(tdme)(D-pen)3}2] by Co L_2,3-edge soft X-ray absorption (XAS) spectroscopy, where PPh3 denotes triphenylphosphine and tdme stands for 1,1,1-tris[(diphenylphosphino)methyl]ethane. The Co L_2,3-edge XAS spectra indicate the localized ionic 3d electronic states in both materials. The experimental spectra are well explained by spectral simulation for a localized Co ion under ligand fields with the full multiplet theory, which verifies that the ions are in the low-spin Co3+ state in the former compound and in the high-spin Co2+ state in the latter.
△ Less
Submitted 21 May, 2017; v1 submitted 15 March, 2016;
originally announced March 2016.
