-
Deep Active Learning in the Presence of Label Noise: A Survey
Authors:
Moseli Mots'oehli,
Kyungim Baek
Abstract:
Deep active learning has emerged as a powerful tool for training deep learning models within a predefined labeling budget. These models have achieved performances comparable to those trained in an offline setting. However, deep active learning faces substantial issues when dealing with classification datasets containing noisy labels. In this literature review, we discuss the current state of deep…
▽ More
Deep active learning has emerged as a powerful tool for training deep learning models within a predefined labeling budget. These models have achieved performances comparable to those trained in an offline setting. However, deep active learning faces substantial issues when dealing with classification datasets containing noisy labels. In this literature review, we discuss the current state of deep active learning in the presence of label noise, highlighting unique approaches, their strengths, and weaknesses. With the recent success of vision transformers in image classification tasks, we provide a brief overview and consider how the transformer layers and attention mechanisms can be used to enhance diversity, importance, and uncertainty-based selection in queries sent to an oracle for labeling. We further propose exploring contrastive learning methods to derive good image representations that can aid in selecting high-value samples for labeling in an active learning setting. We also highlight the need for creating unified benchmarks and standardized datasets for deep active learning in the presence of label noise for image classification to promote the reproducibility of research. The review concludes by suggesting avenues for future research in this area.
△ Less
Submitted 19 September, 2023; v1 submitted 21 February, 2023;
originally announced February 2023.
-
Commonality in Natural Images Rescues GANs: Pretraining GANs with Generic and Privacy-free Synthetic Data
Authors:
Kyungjune Baek,
Hyunjung Shim
Abstract:
Transfer learning for GANs successfully improves generation performance under low-shot regimes. However, existing studies show that the pretrained model using a single benchmark dataset is not generalized to various target datasets. More importantly, the pretrained model can be vulnerable to copyright or privacy risks as membership inference attack advances. To resolve both issues, we propose an e…
▽ More
Transfer learning for GANs successfully improves generation performance under low-shot regimes. However, existing studies show that the pretrained model using a single benchmark dataset is not generalized to various target datasets. More importantly, the pretrained model can be vulnerable to copyright or privacy risks as membership inference attack advances. To resolve both issues, we propose an effective and unbiased data synthesizer, namely Primitives-PS, inspired by the generic characteristics of natural images. Specifically, we utilize 1) the generic statistics on the frequency magnitude spectrum, 2) the elementary shape (i.e., image composition via elementary shapes) for representing the structure information, and 3) the existence of saliency as prior. Since our synthesizer only considers the generic properties of natural images, the single model pretrained on our dataset can be consistently transferred to various target datasets, and even outperforms the previous methods pretrained with the natural images in terms of Fr'echet inception distance. Extensive analysis, ablation study, and evaluations demonstrate that each component of our data synthesizer is effective, and provide insights on the desirable nature of the pretrained model for the transferability of GANs.
△ Less
Submitted 11 April, 2022;
originally announced April 2022.
-
Rethinking the Truly Unsupervised Image-to-Image Translation
Authors:
Kyungjune Baek,
Yunjey Choi,
Youngjung Uh,
Jaejun Yoo,
Hyunjung Shim
Abstract:
Every recent image-to-image translation model inherently requires either image-level (i.e. input-output pairs) or set-level (i.e. domain labels) supervision. However, even set-level supervision can be a severe bottleneck for data collection in practice. In this paper, we tackle image-to-image translation in a fully unsupervised setting, i.e., neither paired images nor domain labels. To this end, w…
▽ More
Every recent image-to-image translation model inherently requires either image-level (i.e. input-output pairs) or set-level (i.e. domain labels) supervision. However, even set-level supervision can be a severe bottleneck for data collection in practice. In this paper, we tackle image-to-image translation in a fully unsupervised setting, i.e., neither paired images nor domain labels. To this end, we propose a truly unsupervised image-to-image translation model (TUNIT) that simultaneously learns to separate image domains and translates input images into the estimated domains. Experimental results show that our model achieves comparable or even better performance than the set-level supervised model trained with full labels, generalizes well on various datasets, and is robust against the choice of hyperparameters (e.g. the preset number of pseudo domains). Furthermore, TUNIT can be easily extended to semi-supervised learning with a few labeled data.
△ Less
Submitted 19 August, 2021; v1 submitted 11 June, 2020;
originally announced June 2020.
-
Editable Generative Adversarial Networks: Generating and Editing Faces Simultaneously
Authors:
Kyungjune Baek,
Duhyeon Bang,
Hyunjung Shim
Abstract:
We propose a novel framework for simultaneously generating and manipulating the face images with desired attributes. While the state-of-the-art attribute editing technique has achieved the impressive performance for creating realistic attribute effects, they only address the image editing problem, using the input image as the condition of model. Recently, several studies attempt to tackle both nov…
▽ More
We propose a novel framework for simultaneously generating and manipulating the face images with desired attributes. While the state-of-the-art attribute editing technique has achieved the impressive performance for creating realistic attribute effects, they only address the image editing problem, using the input image as the condition of model. Recently, several studies attempt to tackle both novel face generation and attribute editing problem using a single solution. However, their image quality is still unsatisfactory. Our goal is to develop a single unified model that can simultaneously create and edit high quality face images with desired attributes. A key idea of our work is that we decompose the image into the latent and attribute vector in low dimensional representation, and then utilize the GAN framework for map** the low dimensional representation to the image. In this way, we can address both the generation and editing problem by learning the generator. For qualitative and quantitative evaluations, the proposed algorithm outperforms recent algorithms addressing the same problem. Also, we show that our model can achieve the competitive performance with the state-of-the-art attribute editing technique in terms of attribute editing quality.
△ Less
Submitted 19 July, 2018;
originally announced July 2018.
-
Performance Analysis of License Assisted Access LTE with Asymmetric Hidden Terminals
Authors:
H. R. Lee,
H. Kim,
H. J. Yang,
J. T. Kim,
S. K. Baek
Abstract:
License Assisted Access (LAA) LTE (LTE-LAA) is a new type of LTE that aggregates the licensed LTE bands with the unlicensed bands via carrier aggregation. To operate in unlicensed bands, LTE-LAA adopts the listen-before-talk policy and designs its channel access mechanism similar to WLAN's DCF. In this paper, we consider an LAA network consisting of an LTE-LAA eNB coexisting with a Wi-Fi STA, and…
▽ More
License Assisted Access (LAA) LTE (LTE-LAA) is a new type of LTE that aggregates the licensed LTE bands with the unlicensed bands via carrier aggregation. To operate in unlicensed bands, LTE-LAA adopts the listen-before-talk policy and designs its channel access mechanism similar to WLAN's DCF. In this paper, we consider an LAA network consisting of an LTE-LAA eNB coexisting with a Wi-Fi STA, and capture the {\em asymmetric hidden terminal problem} where the eNB recognizes the STA while the opposite is not true, which is caused by the asymmetric CCA thresholds between them. We model the network as a joint Markov chain (MC) consisting of two individual MCs, and derive its steady-state probabilities, throughput, and channel access delay along with other key metrics like transmit, busy, collision, and doubling probabilities. Through extensive evaluation, we confirm that the proposed model well predicts the dynamics of the LAA network, and identify important design guidelines for fair coexistence between LTE-LAA and WLAN as follows. First, LTE-LAA should design its contention window (CW) doubling policy by considering Wi-Fi's packet duration and subframe-dependent collision probabilities. Second, there exists a tradeoff between throughput and channel access delay, according to which the CW doubling policy should be adapted.
△ Less
Submitted 13 December, 2016;
originally announced December 2016.
-
Dworkin's Paradox
Authors:
Seung Ki Baek,
Jung-Kyoo Choi,
Beom Jun Kim
Abstract:
How to distribute welfare in a society is a key issue in the subject of distributional justice, which is deeply involved with notions of fairness. Following a thought experiment by Dworkin, this work considers a society of individuals with different preferences on the welfare distribution and an official to mediate the coordination among them. Based on a simple assumption that an individual's welf…
▽ More
How to distribute welfare in a society is a key issue in the subject of distributional justice, which is deeply involved with notions of fairness. Following a thought experiment by Dworkin, this work considers a society of individuals with different preferences on the welfare distribution and an official to mediate the coordination among them. Based on a simple assumption that an individual's welfare is proportional to how her preference is fulfilled by the actual distribution, we show that an egalitarian preference is a strict Nash equilibrium and can be favorable even in certain inhomogeneous situations. These suggest how communication can encourage and secure a notion of fairness.
△ Less
Submitted 28 June, 2012;
originally announced June 2012.
-
The Ten Thousand Kims
Authors:
Seung Ki Baek,
Petter Minnhagen,
Beom Jun Kim
Abstract:
In the Korean culture the family members are recorded in special family books. This makes it possible to follow the distribution of Korean family names far back in history. It is here shown that these name distributions are well described by a simple null model, the random group formation (RGF) model. This model makes it possible to predict how the name distributions change and these predictions a…
▽ More
In the Korean culture the family members are recorded in special family books. This makes it possible to follow the distribution of Korean family names far back in history. It is here shown that these name distributions are well described by a simple null model, the random group formation (RGF) model. This model makes it possible to predict how the name distributions change and these predictions are shown to be borne out. In particular, the RGF model predicts that, for married women entering a collection of family books in a certain year, the occurrence of the most common family name "Kim" should be directly proportional the total number of married women with the same proportionality constant for all the years. This prediction is also borne out to high degree. We speculate that it reflects some inherent social stability in the Korean culture. In addition, we obtain an estimate of the total population of the Korean culture down to year 500 AD, based on the RGF model and find about ten thousand Kims.
△ Less
Submitted 28 September, 2011;
originally announced September 2011.
-
Zipf's law unzipped
Authors:
Seung Ki Baek,
Sebastian Bernhardsson,
Petter Minnhagen
Abstract:
Why does Zipf's law give a good description of data from seemingly completely unrelated phenomena? Here it is argued that the reason is that they can all be described as outcomes of a ubiquitous random group division: the elements can be citizens of a country and the groups family names, or the elements can be all the words making up a novel and the groups the unique words, or the elements could b…
▽ More
Why does Zipf's law give a good description of data from seemingly completely unrelated phenomena? Here it is argued that the reason is that they can all be described as outcomes of a ubiquitous random group division: the elements can be citizens of a country and the groups family names, or the elements can be all the words making up a novel and the groups the unique words, or the elements could be inhabitants and the groups the cities in a country, and so on. A Random Group Formation (RGF) is presented from which a Bayesian estimate is obtained based on minimal information: it provides the best prediction for the number of groups with $k$ elements, given the total number of elements, groups, and the number of elements in the largest group. For each specification of these three values, the RGF predicts a unique group distribution $N(k)\propto \exp(-bk)/k^γ$, where the power-law index $γ$ is a unique function of the same three values. The universality of the result is made possible by the fact that no system specific assumptions are made about the mechanism responsible for the group division. The direct relation between $γ$ and the total number of elements, groups, and the number of elements in the largest group, is calculated. The predictive power of the RGF model is demonstrated by direct comparison with data from a variety of systems. It is shown that $γ$ usually takes values in the interval $1\leqγ\leq 2$ and that the value for a given phenomena depends in a systematic way on the total size of the data set. The results are put in the context of earlier discussions on Zipf's and Gibrat's laws, $N(k)\propto k^{-2}$ and the connection between growth models and RGF is elucidated.
△ Less
Submitted 10 April, 2011;
originally announced April 2011.
-
A Paradoxical Property of the Monkey Book
Authors:
Sebastian Bernhardsson,
Seung Ki Baek,
Petter Minnhagen
Abstract:
A "monkey book" is a book consisting of a random distribution of letters and blanks, where a group of letters surrounded by two blanks is defined as a word. We compare the statistics of the word distribution for a monkey book with the corresponding distribution for the general class of random books, where the latter are books for which the words are randomly distributed. It is shown that the word…
▽ More
A "monkey book" is a book consisting of a random distribution of letters and blanks, where a group of letters surrounded by two blanks is defined as a word. We compare the statistics of the word distribution for a monkey book with the corresponding distribution for the general class of random books, where the latter are books for which the words are randomly distributed. It is shown that the word distribution statistics for the monkey book is different and quite distinct from a typical sampled book or real book. In particular the monkey book obeys Heaps' power law to an extraordinary good approximation, in contrast to the word distributions for sampled and real books, which deviate from Heaps' law in a characteristics way. The somewhat counter-intuitive conclusion is that a "monkey book" obeys Heaps' power law precisely because its word-frequency distribution is not a smooth power law, contrary to the expectation based on simple mathematical arguments that if one is a power law, so is the other.
△ Less
Submitted 14 March, 2011;
originally announced March 2011.
-
Equilibrium solution to the lowest unique positive integer game
Authors:
Seung Ki Baek,
Sebastian Bernhardsson
Abstract:
We address the equilibrium concept of a reverse auction game so that no one can enhance the individual payoff by a unilateral change when all the others follow a certain strategy. In this approach the combinatorial possibilities to consider become very much involved even for a small number of players, which has hindered a precise analysis in previous works. We here present a systematic way to re…
▽ More
We address the equilibrium concept of a reverse auction game so that no one can enhance the individual payoff by a unilateral change when all the others follow a certain strategy. In this approach the combinatorial possibilities to consider become very much involved even for a small number of players, which has hindered a precise analysis in previous works. We here present a systematic way to reach the solution for a general number of players, and show that this game is an example of conflict between the group and the individual interests.
△ Less
Submitted 7 January, 2010;
originally announced January 2010.
