-
Teacher-Explorer-Student Learning: A Novel Learning Method for Open Set Recognition
Authors:
Jaeyeon Jang,
Chang Ouk Kim
Abstract:
If an unknown example that is not seen during training appears, most recognition systems usually produce overgeneralized results and determine that the example belongs to one of the known classes. To address this problem, teacher-explorer-student (T/E/S) learning, which adopts the concept of open set recognition (OSR) that aims to reject unknown samples while minimizing the loss of classification…
▽ More
If an unknown example that is not seen during training appears, most recognition systems usually produce overgeneralized results and determine that the example belongs to one of the known classes. To address this problem, teacher-explorer-student (T/E/S) learning, which adopts the concept of open set recognition (OSR) that aims to reject unknown samples while minimizing the loss of classification performance on known samples, is proposed in this study. In this novel learning method, overgeneralization of deep learning classifiers is significantly reduced by exploring various possibilities of unknowns. Here, the teacher network extracts some hints about unknowns by distilling the pretrained knowledge about knowns and delivers this distilled knowledge to the student. After learning the distilled knowledge, the student network shares the learned information with the explorer network. Then, the explorer network shares its exploration results by generating unknown-like samples and feeding the samples to the student network. By repeating this alternating learning process, the student network experiences a variety of synthetic unknowns, reducing overgeneralization. Extensive experiments were conducted, and the experimental results showed that each component proposed in this paper significantly contributes to the improvement in OSR performance. As a result, the proposed T/E/S learning method outperformed current state-of-the-art methods.
△ Less
Submitted 23 March, 2021;
originally announced March 2021.
-
Collective Decision of One-vs-Rest Networks for Open Set Recognition
Authors:
Jaeyeon Jang,
Chang Ouk Kim
Abstract:
Unknown examples that are unseen during training often appear in real-world machine learning tasks, and an intelligent self-learning system should be able to distinguish between known and unknown examples. Accordingly, open set recognition (OSR), which addresses the problem of classifying knowns and identifying unknowns, has recently been highlighted. However, conventional deep neural networks usi…
▽ More
Unknown examples that are unseen during training often appear in real-world machine learning tasks, and an intelligent self-learning system should be able to distinguish between known and unknown examples. Accordingly, open set recognition (OSR), which addresses the problem of classifying knowns and identifying unknowns, has recently been highlighted. However, conventional deep neural networks using a softmax layer are vulnerable to overgeneralization, producing high confidence scores for unknowns. In this paper, we propose a simple OSR method based on the intuition that OSR performance can be maximized by setting strict and sophisticated decision boundaries that reject unknowns while maintaining satisfactory classification performance on knowns. For this purpose, a novel network structure is proposed, in which multiple one-vs-rest networks (OVRNs) follow a convolutional neural network feature extractor. Here, the OVRN is a simple feed-forward neural network that enhances the ability to reject nonmatches by learning class-specific discriminative features. Furthermore, the collective decision score is modeled by combining the multiple decisions reached by the OVRNs to alleviate overgeneralization. Extensive experiments were conducted on various datasets, and the experimental results showed that the proposed method performed significantly better than the state-of-the-art methods by effectively reducing overgeneralization.
△ Less
Submitted 19 March, 2021; v1 submitted 18 March, 2021;
originally announced March 2021.
-
Freudian and Newtonian Recurrent Cell for Sequential Recommendation
Authors:
Hoyeop Lee,
**bae Im,
Chang Ouk Kim,
Sehee Chung
Abstract:
A sequential recommender system aims to recommend attractive items to users based on behaviour patterns. The predominant sequential recommendation models are based on natural language processing models, such as the gated recurrent unit, that embed items in some defined space and grasp the user's long-term and short-term preferences based on the item embeddings. However, these approaches lack funda…
▽ More
A sequential recommender system aims to recommend attractive items to users based on behaviour patterns. The predominant sequential recommendation models are based on natural language processing models, such as the gated recurrent unit, that embed items in some defined space and grasp the user's long-term and short-term preferences based on the item embeddings. However, these approaches lack fundamental insight into how such models are related to the user's inherent decision-making process. To provide this insight, we propose a novel recurrent cell, namely FaNC, from Freudian and Newtonian perspectives. FaNC divides the user's state into conscious and unconscious states, and the user's decision process is modelled by Freud's two principles: the pleasure principle and reality principle. To model the pleasure principle, i.e., free-floating user's instinct, we place the user's unconscious state and item embeddings in the same latent space and subject them to Newton's law of gravitation. Moreover, to recommend items to users, we model the reality principle, i.e., balancing the conscious and unconscious states, via a gating function. Based on extensive experiments on various benchmark datasets, this paper provides insight into the characteristics of the proposed model. FaNC initiates a new direction of sequential recommendations at the convergence of psychoanalysis and recommender systems.
△ Less
Submitted 11 February, 2021;
originally announced February 2021.
-
One-vs-Rest Network-based Deep Probability Model for Open Set Recognition
Authors:
Jaeyeon Jang,
Chang Ouk Kim
Abstract:
Unknown examples that are unseen during training often appear in real-world computer vision tasks, and an intelligent self-learning system should be able to differentiate between known and unknown examples. Open set recognition, which addresses this problem, has been studied for approximately a decade. However, conventional open set recognition methods based on deep neural networks (DNNs) lack a f…
▽ More
Unknown examples that are unseen during training often appear in real-world computer vision tasks, and an intelligent self-learning system should be able to differentiate between known and unknown examples. Open set recognition, which addresses this problem, has been studied for approximately a decade. However, conventional open set recognition methods based on deep neural networks (DNNs) lack a foundation for post recognition score analysis. In this paper, we propose a DNN structure in which multiple one-vs-rest sigmoid networks follow a convolutional neural network feature extractor. A one-vs-rest network, which is composed of rectified linear unit activation functions for the hidden layers and a single sigmoid target class output node, can maximize the ability to learn information from nonmatch examples. Furthermore, the network yields a sophisticated nonlinear features-to-output map** that is explainable in the feature space. By introducing extreme value theory-based calibration techniques, the nonlinear and explainable map** provides a well-grounded class membership probability models. Our experiments show that one-vs-rest networks can provide more informative hidden representations for unknown examples than the commonly used SoftMax layer. In addition, the proposed probability model outperformed the state-of-the art methods in open set classification scenarios.
△ Less
Submitted 28 May, 2020; v1 submitted 17 April, 2020;
originally announced April 2020.
-
Amplifying the Imitation Effect for Reinforcement Learning of UCAV's Mission Execution
Authors:
Gyeong Taek Lee,
Chang Ouk Kim
Abstract:
This paper proposes a new reinforcement learning (RL) algorithm that enhances exploration by amplifying the imitation effect (AIE). This algorithm consists of self-imitation learning and random network distillation algorithms. We argue that these two algorithms complement each other and that combining these two algorithms can amplify the imitation effect for exploration. In addition, by adding an…
▽ More
This paper proposes a new reinforcement learning (RL) algorithm that enhances exploration by amplifying the imitation effect (AIE). This algorithm consists of self-imitation learning and random network distillation algorithms. We argue that these two algorithms complement each other and that combining these two algorithms can amplify the imitation effect for exploration. In addition, by adding an intrinsic penalty reward to the state that the RL agent frequently visits and using replay memory for learning the feature state when using an exploration bonus, the proposed approach leads to deep exploration and deviates from the current converged policy. We verified the exploration performance of the algorithm through experiments in a two-dimensional grid environment. In addition, we applied the algorithm to a simulated environment of unmanned combat aerial vehicle (UCAV) mission execution, and the empirical results show that AIE is very effective for finding the UCAV's shortest flight path to avoid an enemy's missiles.
△ Less
Submitted 17 January, 2019;
originally announced January 2019.
-
Modeling of torsion stress giant magnetoimpedance in amorphous wires with negative magnetostriction
Authors:
N. A. Buznikov,
C. O. Kim
Abstract:
A model describing the influence of torsion stress on the giant magnetoimpedance in amorphous wires with negative magnetostriction is proposed. The wire impedance is found by means of the solution of Maxwell equations together with the Landau-Lifshitz equation, assuming a simplified spatial distribution of the magnetoelastic anisotropy induced by the torsion stress. The impedance is analyzed as…
▽ More
A model describing the influence of torsion stress on the giant magnetoimpedance in amorphous wires with negative magnetostriction is proposed. The wire impedance is found by means of the solution of Maxwell equations together with the Landau-Lifshitz equation, assuming a simplified spatial distribution of the magnetoelastic anisotropy induced by the torsion stress. The impedance is analyzed as a function of the external magnetic field, torsion stress and frequency. It is shown that the magnetoimpedance ratio torsion dependence has an asymmetric shape, with a sharp peak at some value of the torsion stress. The calculated field and stress dependences of the impedance are in qualitative agreement with results of the experimental study of the torsion stress giant magnetoimpedance in Co-based amorphous wires.
△ Less
Submitted 1 May, 2007;
originally announced May 2007.
-
A model for exchange-biased asymmetric giant magneto-impedance in amorphous wires
Authors:
N. A. Buznikov,
S. S. Yoon,
C. G. Kim,
C. O. Kim
Abstract:
A model describing the exchange-biased asymmetric giant magneto-impedance in Joule-heated amorphous wires is proposed. It is assumed that the Joule heating results in the formation of a surface hard magnetic crystalline layer, and the asymmetric giant magneto-impedance is related to the exchange coupling between the amorphous and crystalline phases. The coupling between the surface layer and the…
▽ More
A model describing the exchange-biased asymmetric giant magneto-impedance in Joule-heated amorphous wires is proposed. It is assumed that the Joule heating results in the formation of a surface hard magnetic crystalline layer, and the asymmetric giant magneto-impedance is related to the exchange coupling between the amorphous and crystalline phases. The coupling between the surface layer and the amorphous bulk is described in terms of an effective bias field. The model is based on a simultaneous solution of linearizied Maxwell equations and the Landau-Lifshitz equation. The calculated field and frequency dependences of the wire impedance are in a qualitative agreement with results of the experimental study of the asymmetric giant magneto-impedance in Joule-heated Co-based amorphous wires.
△ Less
Submitted 14 July, 2006;
originally announced July 2006.
-
Generation of second harmonic in off-diagonal magneto-impedance in Co-based amorphous ribbons
Authors:
N. A. Buznikov,
S. S. Yoon,
L. **,
C. O Kim,
C. G. Kim
Abstract:
The off-diagonal magneto-impedance in Co-based amorphous ribbons was measured using a pick-up coil wound around the sample. The ribbons were annealed in air or in vacuum in the presence of a weak magnetic field. The evolution of the first and second harmonics in the pick-up coil voltage as a function of the current amplitude was studied. At low current amplitudes, the first harmonic dominates in…
▽ More
The off-diagonal magneto-impedance in Co-based amorphous ribbons was measured using a pick-up coil wound around the sample. The ribbons were annealed in air or in vacuum in the presence of a weak magnetic field. The evolution of the first and second harmonics in the pick-up coil voltage as a function of the current amplitude was studied. At low current amplitudes, the first harmonic dominates in the frequency spectrum of the voltage, and at sufficiently high current amplitudes, the amplitude of the second harmonic becomes higher than that of the first harmonic. For air-annealed ribbons, the asymmetric two-peak behaviour of the field dependences of the harmonic amplitudes was observed, which is related to the coupling between the amorphous phase and surface crystalline layers appearing after annealing. For vacuum-annealed samples, the first harmonic has a maximum at zero external field, and the field dependence of the second harmonic exhibits symmetric two-peak behaviour. The experimental results are interpreted in terms of a quasi-static rotational model. It is shown that the appearance of the second harmonic in the pick-up coil voltage is related to the anti-symmetrical distribution of the transverse field induced by the current. The calculated dependences are in a qualitative agreement with the experimental data.
△ Less
Submitted 16 December, 2005;
originally announced December 2005.
-
Giant magnetoimpedance in composite wires with insulator layer between non-magnetic core and soft magnetic shell
Authors:
N. A. Buznikov,
A. S. Antonov,
A. B. Granovsky,
C. G. Kim,
C. O. Kim,
X. P. Li,
S. S. Yoon
Abstract:
A method for calculation of the magnetoimpedance in composite wires having an insulator layer between non-magnetic core and soft magnetic shell is described. It is assumed that the magnetic shell has a helical anisotropy and the driving current flows through the core only. The distribution of eddy currents and expressions for the impedance are found by means of a solution of Maxwell equations ta…
▽ More
A method for calculation of the magnetoimpedance in composite wires having an insulator layer between non-magnetic core and soft magnetic shell is described. It is assumed that the magnetic shell has a helical anisotropy and the driving current flows through the core only. The distribution of eddy currents and expressions for the impedance are found by means of a solution of Maxwell equations taking into account the magnetization dynamics within the shell governed by the Landau-Lifshitz equation. The effect of the insulator layer on the magnetoimpedance is analyzed.
△ Less
Submitted 16 November, 2005;
originally announced November 2005.
-
Off-diagonal magnetoimpedance in field-annealed Co-based amorphous ribbons
Authors:
N. A. Buznikov,
C. G. Kim,
C. O. Kim,
L. **,
S. S. Yoon
Abstract:
The off-diagonal magnetoimpedance in field-annealed CoFeSiB amorphous ribbons was measured in the low-frequency range using a pick-up coil wound around the sample. The asymmetric two-peak behavior of the field dependence of the off-diagonal impedance was observed. The asymmetry is attributed to the formation of a hard magnetic crystalline phase at the ribbon surface. The experimental results are…
▽ More
The off-diagonal magnetoimpedance in field-annealed CoFeSiB amorphous ribbons was measured in the low-frequency range using a pick-up coil wound around the sample. The asymmetric two-peak behavior of the field dependence of the off-diagonal impedance was observed. The asymmetry is attributed to the formation of a hard magnetic crystalline phase at the ribbon surface. The experimental results are interpreted in terms of the surface impedance tensor. It is assumed that the ribbon consists of an inner amorphous region and surface crystalline layers. The coupling between the crystalline and amorphous phases is described through an effective bias field. A qualitative agreement between the calculated dependences and experimental data is demonstrated. The results obtained may be useful for development of weak magnetic-field sensors.
△ Less
Submitted 14 December, 2005; v1 submitted 31 March, 2005;
originally announced March 2005.
-
Modeling of asymmetric giant magnetoimpedance in amorphous ribbons with a surface crystalline layer
Authors:
N. A. Buznikov,
C. G. Kim,
C. O. Kim,
S. S. Yoon
Abstract:
A model describing the asymmetric giant magnetoimpedance (GMI) in field-annealed amorphous ribbons is proposed. It is assumed that the ribbon consists of an inner amorphous core and surface hard magnetic crystalline layers. The model is based on a simultaneous solution of linearizied Maxwell equations and Landau-Lifshitz equation. The coupling between the surface layers and the amorphous core is…
▽ More
A model describing the asymmetric giant magnetoimpedance (GMI) in field-annealed amorphous ribbons is proposed. It is assumed that the ribbon consists of an inner amorphous core and surface hard magnetic crystalline layers. The model is based on a simultaneous solution of linearizied Maxwell equations and Landau-Lifshitz equation. The coupling between the surface layers and the amorphous core is described in terms of an effective bias field. Analytical expressions for the frequency and field dependences of the ribbon impedance are found. The calculated dependences are in a qualitative agreement with results of experimental studies of the high-frequency asymmetric giant GMI in field-annealed amorphous ribbons.
△ Less
Submitted 4 March, 2005;
originally announced March 2005.
-
A model for asymmetric giant magnetoimpedance in field-annealed amorphous ribbons
Authors:
N. A. Buznikov,
C. G. Kim,
C. O. Kim,
S. S. Yoon
Abstract:
A phenomenological model for the asymmetric giant magnetoimpedance (GMI) in field-annealed amorphous ribbons is developed. The effect of a surface crystalline layer on the GMI response is described in terms of an effective bias field appearing due to a coupling between the crystalline layer and amorphous phase. It is shown that the presence of the bias field changes drastically the GMI profile.…
▽ More
A phenomenological model for the asymmetric giant magnetoimpedance (GMI) in field-annealed amorphous ribbons is developed. The effect of a surface crystalline layer on the GMI response is described in terms of an effective bias field appearing due to a coupling between the crystalline layer and amorphous phase. It is shown that the presence of the bias field changes drastically the GMI profile. At low frequencies, the domain-walls motion leads to a steplike change in the GMI response. At high frequencies, the domain-walls motion is damped, and the GMI profile exhibits asymmetric two-peak behavior. The calculated dependences are shown to be in a qualitative agreement with results of experimental studies of the asymmetric GMI in field-annealed Co-based amorphous ribbons.
△ Less
Submitted 14 February, 2005;
originally announced February 2005.
-
Current distribution and giant magnetoimpedance in composite wires with helical magnetic anisotropy
Authors:
N. A. Buznikov,
A. S. Antonov,
A. B. Granovsky,
C. G. Kim,
C. O. Kim,
X. P. Li,
S. S. Yoon
Abstract:
The giant magnetoimpedance effect in composite wires consising of a non-magnetic inner core and soft magnetic shell is studied theoretically. It is assumed that the magnetic shell has a helical anisotropy. The current and field distributions in the composite wire are found by means of a simultaneous solution of Maxwell equations and the Landau-Lifshitz equation. The expressions for the diagonal…
▽ More
The giant magnetoimpedance effect in composite wires consising of a non-magnetic inner core and soft magnetic shell is studied theoretically. It is assumed that the magnetic shell has a helical anisotropy. The current and field distributions in the composite wire are found by means of a simultaneous solution of Maxwell equations and the Landau-Lifshitz equation. The expressions for the diagonal and off-diagonal impedance are obtained for low and high frequencies. The dependences of the impedance on the anisotropy axis angle and the shell thickness are analyzed. Maximum field sensitivity is shown to correspond to the case of the circular anisotropy in the magnetic shell. It is demonstrated that the optimum shell thickness to obtain maximum impedance ratio is equal to the effective skin depth in the mahnetic material.
△ Less
Submitted 20 October, 2005; v1 submitted 3 February, 2005;
originally announced February 2005.
