LassoLayer: Nonlinear Feature Selection by Switching One-to-one Links
Authors:
Akihito Sudo,
Teng Teck Hou,
Masaki Yamaguchi,
Yoshinori Tone
Abstract:
Along with the desire to address more complex problems, feature selection methods have gained in importance. Feature selection methods can be classified into wrapper method, filter method, and embedded method. Being a powerful embedded feature selection method, Lasso has attracted the attention of many researchers. However, as a linear approach, the applicability of Lasso has been limited. In this…
▽ More
Along with the desire to address more complex problems, feature selection methods have gained in importance. Feature selection methods can be classified into wrapper method, filter method, and embedded method. Being a powerful embedded feature selection method, Lasso has attracted the attention of many researchers. However, as a linear approach, the applicability of Lasso has been limited. In this work, we propose LassoLayer that is one-to-one connected and trained by L1 optimization, which work to drop out unnecessary units for prediction. For nonlinear feature selections, we build LassoMLP: the network equipped with LassoLayer as its first layer. Because we can insert LassoLayer in any network structure, it can harness the strength of neural network suitable for tasks where feature selection is needed. We evaluate LassoMLP in feature selection with regression and classification tasks. LassoMLP receives features including considerable numbers of noisy factors that is harmful for overfitting. In the experiments using MNIST dataset, we confirm that LassoMLP outperforms the state-of-the-art method.
△ Less
Submitted 27 August, 2021;
originally announced August 2021.
Thermodynamics of Internal Correlations
Authors:
Akihito Sudo
Abstract:
Previous research has consistently affirmed that Maxwell's demon must adhere to the second law of thermodynamics. Yet, the unresolved question remains whether the profitability and indispensability of information, which we routinely take for granted, are based on constraints stemming from physical laws. This paper reports a novel generalization of the second law of thermodynamics, answering that w…
▽ More
Previous research has consistently affirmed that Maxwell's demon must adhere to the second law of thermodynamics. Yet, the unresolved question remains whether the profitability and indispensability of information, which we routinely take for granted, are based on constraints stemming from physical laws. This paper reports a novel generalization of the second law of thermodynamics, answering that when internal correlations, i.e., correlations between subsystems of resource, are intended to be exploited, information is indispensable to extract free energy. Furthermore, the internal correlations, which can grow linearly with the number of subsystems in the resource, allow for control with information that yields significant gains, dwarfing the negligible operational costs in the thermodynamic limit. Thus, the generalized second law presented herein can be interpreted as a fundamental physical principle that ensures the benefit and inevitability of information processing in thermodynamics.
△ Less
Submitted 13 December, 2023; v1 submitted 2 August, 2021;
originally announced August 2021.
