Showing 1–16 of 16 results for author: Yoon, J W

Speed-up of Data Analysis with Kernel Trick in Encrypted Domain

Authors: Joon Soo Yoo, Baek Kyung Song, Tae Min Ahn, Ji Won Heo, Ji Won Yoon

Abstract: Homomorphic encryption (HE) is pivotal for secure computation on encrypted data, crucial in privacy-preserving data analysis. However, efficiently processing high-dimensional data in HE, especially for machine learning and statistical (ML/STAT) algorithms, poses a challenge. In this paper, we present an effective acceleration method using the kernel method for HE schemes, enhancing time performanc… ▽ More Homomorphic encryption (HE) is pivotal for secure computation on encrypted data, crucial in privacy-preserving data analysis. However, efficiently processing high-dimensional data in HE, especially for machine learning and statistical (ML/STAT) algorithms, poses a challenge. In this paper, we present an effective acceleration method using the kernel method for HE schemes, enhancing time performance in ML/STAT algorithms within encrypted domains. This technique, independent of underlying HE mechanisms and complementing existing optimizations, notably reduces costly HE multiplications, offering near constant time complexity relative to data dimension. Aimed at accessibility, this method is tailored for data scientists and developers with limited cryptography background, facilitating advanced data analysis in secure environments. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: Submitted as a preprint

Self-evolving Autoencoder Embedded Q-Network

Authors: J. Senthilnath, Bangjian Zhou, Zhen Wei Ng, Deeksha Aggarwal, Rajdeep Dutta, Ji Wei Yoon, Aye Phyu Phyu Aung, Keyu Wu, Min Wu, Xiaoli Li

Abstract: In the realm of sequential decision-making tasks, the exploration capability of a reinforcement learning (RL) agent is paramount for achieving high rewards through interactions with the environment. To enhance this crucial ability, we propose SAQN, a novel approach wherein a self-evolving autoencoder (SA) is embedded with a Q-Network (QN). In SAQN, the self-evolving autoencoder architecture adapts… ▽ More In the realm of sequential decision-making tasks, the exploration capability of a reinforcement learning (RL) agent is paramount for achieving high rewards through interactions with the environment. To enhance this crucial ability, we propose SAQN, a novel approach wherein a self-evolving autoencoder (SA) is embedded with a Q-Network (QN). In SAQN, the self-evolving autoencoder architecture adapts and evolves as the agent explores the environment. This evolution enables the autoencoder to capture a diverse range of raw observations and represent them effectively in its latent space. By leveraging the disentangled states extracted from the encoder generated latent space, the QN is trained to determine optimal actions that improve rewards. During the evolution of the autoencoder architecture, a bias-variance regulatory strategy is employed to elicit the optimal response from the RL agent. This strategy involves two key components: (i) fostering the growth of nodes to retain previously acquired knowledge, ensuring a rich representation of the environment, and (ii) pruning the least contributing nodes to maintain a more manageable and tractable latent space. Extensive experimental evaluations conducted on three distinct benchmark environments and a real-world molecular environment demonstrate that the proposed SAQN significantly outperforms state-of-the-art counterparts. The results highlight the effectiveness of the self-evolving autoencoder and its collaboration with the Q-Network in tackling sequential decision-making tasks. △ Less

Submitted 18 February, 2024; originally announced February 2024.

Comments: 11 pages, 9 figures, 3 tables

Explainable machine learning to enable high-throughput electrical conductivity optimization and discovery of doped conjugated polymers

Authors: Ji Wei Yoon, Adithya Kumar, Pawan Kumar, Kedar Hippalgaonkar, J Senthilnath, Vijila Chellappan

Abstract: The combination of high-throughput experimentation techniques and machine learning (ML) has recently ushered in a new era of accelerated material discovery, enabling the identification of materials with cutting-edge properties. However, the measurement of certain physical quantities remains challenging to automate. Specifically, meticulous process control, experimentation and laborious measurement… ▽ More The combination of high-throughput experimentation techniques and machine learning (ML) has recently ushered in a new era of accelerated material discovery, enabling the identification of materials with cutting-edge properties. However, the measurement of certain physical quantities remains challenging to automate. Specifically, meticulous process control, experimentation and laborious measurements are required to achieve optimal electrical conductivity in doped polymer materials. We propose a ML approach, which relies on readily measured absorbance spectra, to accelerate the workflow associated with measuring electrical conductivity. The classification model accurately classifies samples with a conductivity > 25 to 100 S/cm, achieving a maximum of 100 % accuracy rate. For the subset of highly conductive samples, we employed a regression model to predict their conductivities, yielding an impressive test R2 value of 0.984. We tested the models with samples of the two highest conductivities (498 and 506 S/cm) and showed that they were able to correctly classify and predict the two extrapolative conductivities at satisfactory levels of errors. The proposed ML-assisted workflow results in an improvement in the efficiency of the conductivity measurements by 89 % of the maximum achievable using our experimental techniques. Furthermore, our approach addressed the common challenge of the lack of explainability in ML models by exploiting bespoke mathematical properties of the descriptors and ML model, allowing us to gain corroborated insights into the spectral influences on conductivity. Through this study, we offer an accelerated pathway for optimizing the properties of doped polymer materials while showcasing the valuable insights that can be derived from purposeful utilization of ML in experimental science. △ Less

Submitted 27 April, 2024; v1 submitted 8 August, 2023; originally announced August 2023.

Comments: 33 Pages, 17 figures

Journal ref: Knowledge-Based Systems 295C (2024) 111812

EM-Network: Oracle Guided Self-distillation for Sequence Learning

Authors: Ji Won Yoon, Sunghwan Ahn, Hyeonseung Lee, Minchan Kim, Seok Min Kim, Nam Soo Kim

Abstract: We introduce EM-Network, a novel self-distillation approach that effectively leverages target information for supervised sequence-to-sequence (seq2seq) learning. In contrast to conventional methods, it is trained with oracle guidance, which is derived from the target sequence. Since the oracle guidance compactly represents the target-side context that can assist the sequence model in solving the t… ▽ More We introduce EM-Network, a novel self-distillation approach that effectively leverages target information for supervised sequence-to-sequence (seq2seq) learning. In contrast to conventional methods, it is trained with oracle guidance, which is derived from the target sequence. Since the oracle guidance compactly represents the target-side context that can assist the sequence model in solving the task, the EM-Network achieves a better prediction compared to using only the source input. To allow the sequence model to inherit the promising capability of the EM-Network, we propose a new self-distillation strategy, where the original sequence model can benefit from the knowledge of the EM-Network in a one-stage manner. We conduct comprehensive experiments on two types of seq2seq models: connectionist temporal classification (CTC) for speech recognition and attention-based encoder-decoder (AED) for machine translation. Experimental results demonstrate that the EM-Network significantly advances the current state-of-the-art approaches, improving over the best prior work on speech recognition and establishing state-of-the-art performance on WMT'14 and IWSLT'14. △ Less

Submitted 14 June, 2023; originally announced June 2023.

Comments: ICML 2023

Development of deep biological ages aware of morbidity and mortality based on unsupervised and semi-supervised deep learning approaches

Authors: Seong-Eun Moon, Ji Won Yoon, Shinyoung Joo, Yoohyung Kim, Jae Hyun Bae, Seokho Yoon, Haanju Yoo, Young Min Cho

Abstract: Background: While deep learning technology, which has the capability of obtaining latent representations based on large-scale data, can be a potential solution for the discovery of a novel aging biomarker, existing deep learning methods for biological age estimation usually depend on chronological ages and lack of consideration of mortality and morbidity that are the most significant outcomes of a… ▽ More Background: While deep learning technology, which has the capability of obtaining latent representations based on large-scale data, can be a potential solution for the discovery of a novel aging biomarker, existing deep learning methods for biological age estimation usually depend on chronological ages and lack of consideration of mortality and morbidity that are the most significant outcomes of aging. Methods: This paper proposes a novel deep learning model to learn latent representations of biological aging in regard to subjects' morbidity and mortality. The model utilizes health check-up data in addition to morbidity and mortality information to learn the complex relationships between aging and measured clinical attributes. Findings: The proposed model is evaluated on a large dataset of general populations compared with KDM and other learning-based models. Results demonstrate that biological ages obtained by the proposed model have superior discriminability of subjects' morbidity and mortality. △ Less

Submitted 1 February, 2023; originally announced February 2023.

Inter-KD: Intermediate Knowledge Distillation for CTC-Based Automatic Speech Recognition

Authors: Ji Won Yoon, Beom Jun Woo, Sunghwan Ahn, Hyeonseung Lee, Nam Soo Kim

Abstract: Recently, the advance in deep learning has brought a considerable improvement in the end-to-end speech recognition field, simplifying the traditional pipeline while producing promising results. Among the end-to-end models, the connectionist temporal classification (CTC)-based model has attracted research interest due to its non-autoregressive nature. However, such CTC models require a heavy comput… ▽ More Recently, the advance in deep learning has brought a considerable improvement in the end-to-end speech recognition field, simplifying the traditional pipeline while producing promising results. Among the end-to-end models, the connectionist temporal classification (CTC)-based model has attracted research interest due to its non-autoregressive nature. However, such CTC models require a heavy computational cost to achieve outstanding performance. To mitigate the computational burden, we propose a simple yet effective knowledge distillation (KD) for the CTC framework, namely Inter-KD, that additionally transfers the teacher's knowledge to the intermediate CTC layers of the student network. From the experimental results on the LibriSpeech, we verify that the Inter-KD shows better achievements compared to the conventional KD methods. Without using any language model (LM) and data augmentation, Inter-KD improves the word error rate (WER) performance from 8.85 % to 6.30 % on the test-clean. △ Less

Submitted 28 November, 2022; originally announced November 2022.

Comments: Accepted by 2022 SLT Workshop

HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition

Authors: Ji Won Yoon, Beom Jun Woo, Nam Soo Kim

Abstract: Pre-training with self-supervised models, such as Hidden-unit BERT (HuBERT) and wav2vec 2.0, has brought significant improvements in automatic speech recognition (ASR). However, these models usually require an expensive computational cost to achieve outstanding performance, slowing down the inference speed. To improve the model efficiency, we introduce an early exit scheme for ASR, namely HuBERT-E… ▽ More Pre-training with self-supervised models, such as Hidden-unit BERT (HuBERT) and wav2vec 2.0, has brought significant improvements in automatic speech recognition (ASR). However, these models usually require an expensive computational cost to achieve outstanding performance, slowing down the inference speed. To improve the model efficiency, we introduce an early exit scheme for ASR, namely HuBERT-EE, that allows the model to stop the inference dynamically. In HuBERT-EE, multiple early exit branches are added at the intermediate layers. When the intermediate prediction of the early exit branch is confident, the model stops the inference, and the corresponding result can be returned early. We investigate the proper early exiting criterion and fine-tuning strategy to effectively perform early exiting. Experimental results on the LibriSpeech show that HuBERT-EE can accelerate the inference of the HuBERT while simultaneously balancing the trade-off between the performance and the latency. △ Less

Submitted 19 June, 2024; v1 submitted 13 April, 2022; originally announced April 2022.

Comments: Accepted by INTERSPEECH 2024

Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models

Authors: Ji Won Yoon, Hyung Yong Kim, Hyeonseung Lee, Sunghwan Ahn, Nam Soo Kim

Abstract: Knowledge distillation (KD), best known as an effective method for model compression, aims at transferring the knowledge of a bigger network (teacher) to a much smaller network (student). Conventional KD methods usually employ the teacher model trained in a supervised manner, where output labels are treated only as targets. Extending this supervised scheme further, we introduce a new type of teach… ▽ More Knowledge distillation (KD), best known as an effective method for model compression, aims at transferring the knowledge of a bigger network (teacher) to a much smaller network (student). Conventional KD methods usually employ the teacher model trained in a supervised manner, where output labels are treated only as targets. Extending this supervised scheme further, we introduce a new type of teacher model for connectionist temporal classification (CTC)-based sequence models, namely Oracle Teacher, that leverages both the source inputs and the output labels as the teacher model's input. Since the Oracle Teacher learns a more accurate CTC alignment by referring to the target information, it can provide the student with more optimal guidance. One potential risk for the proposed approach is a trivial solution that the model's output directly copies the target input. Based on a many-to-one map** property of the CTC algorithm, we present a training strategy that can effectively prevent the trivial solution and thus enables utilizing both source and target inputs for model training. Extensive experiments are conducted on two sequence learning tasks: speech recognition and scene text recognition. From the experimental results, we empirically show that the proposed model improves the students across these tasks while achieving a considerable speed-up in the teacher model's training time. △ Less

Submitted 11 August, 2023; v1 submitted 5 November, 2021; originally announced November 2021.

Comments: Accepted by IEEE/ACM Transactions on Audio, Speech and Language Processing

Speech to Text Adaptation: Towards an Efficient Cross-Modal Distillation

Authors: Won Ik Cho, Donghyun Kwak, Ji Won Yoon, Nam Soo Kim

Abstract: Speech is one of the most effective means of communication and is full of information that helps the transmission of utterer's thoughts. However, mainly due to the cumbersome processing of acoustic features, phoneme or word posterior probability has frequently been discarded in understanding the natural language. Thus, some recent spoken language understanding (SLU) modules have utilized end-to-en… ▽ More Speech is one of the most effective means of communication and is full of information that helps the transmission of utterer's thoughts. However, mainly due to the cumbersome processing of acoustic features, phoneme or word posterior probability has frequently been discarded in understanding the natural language. Thus, some recent spoken language understanding (SLU) modules have utilized end-to-end structures that preserve the uncertainty information. This further reduces the propagation of speech recognition error and guarantees computational efficiency. We claim that in this process, the speech comprehension can benefit from the inference of massive pre-trained language models (LMs). We transfer the knowledge from a concrete Transformer-based text LM to an SLU module which can face a data shortage, based on recent cross-modal distillation methodologies. We demonstrate the validity of our proposal upon the performance on Fluent Speech Command, an English SLU benchmark. Thereby, we experimentally verify our hypothesis that the knowledge could be shared from the top layer of the LM to a fully speech-based module, in which the abstracted speech is expected to meet the semantic representation. △ Less

Submitted 8 August, 2020; v1 submitted 17 May, 2020; originally announced May 2020.

Comments: Interspeech 2020 Camera-ready

Speech Intention Understanding in a Head-final Language: A Disambiguation Utilizing Intonation-dependency

Authors: Won Ik Cho, Hyeon Seung Lee, Ji Won Yoon, Seok Min Kim, Nam Soo Kim

Abstract: For a large portion of real-life utterances, the intention cannot be solely decided by either their semantic or syntactic characteristics. Although not all the sociolinguistic and pragmatic information can be digitized, at least phonetic features are indispensable in understanding the spoken language. Especially in head-final languages such as Korean, sentence-final prosody has great importance in… ▽ More For a large portion of real-life utterances, the intention cannot be solely decided by either their semantic or syntactic characteristics. Although not all the sociolinguistic and pragmatic information can be digitized, at least phonetic features are indispensable in understanding the spoken language. Especially in head-final languages such as Korean, sentence-final prosody has great importance in identifying the speaker's intention. This paper suggests a system which identifies the inherent intention of a spoken utterance given its transcript, in some cases using auxiliary acoustic features. The main point here is a separate distinction for cases where discrimination of intention requires an acoustic cue. Thus, the proposed classification system decides whether the given utterance is a fragment, statement, question, command, or a rhetorical question/command, utilizing the intonation-dependency coming from the head-finality. Based on an intuitive understanding of the Korean language that is engaged in the data annotation, we construct a network which identifies the intention of a speech, and validate its utility with the test sentences. The system, if combined with up-to-date speech recognizers, is expected to be flexibly inserted into various language understanding modules. △ Less

Submitted 26 June, 2022; v1 submitted 10 November, 2018; originally announced November 2018.

Comments: 14 pages, 2 figures, 7 tables; Identical to the previous revision. The latest version of this manuscript is recently accepted at ACM TALLIP, with the modified title, authors, and contents (see the DOI below). Please refer to THIS version only when relevant to the analysis with speech data, and refer to the journal version to cite the protocol and dataset

An Efficient Model Selection for Gaussian Mixture Model in a Bayesian Framework

Authors: Ji Won Yoon

Abstract: In order to cluster or partition data, we often use Expectation-and-Maximization (EM) or Variational approximation with a Gaussian Mixture Model (GMM), which is a parametric probability density function represented as a weighted sum of $\hat{K}$ Gaussian component densities. However, model selection to find underlying $\hat{K}$ is one of the key concerns in GMM clustering, since we can obtain the… ▽ More In order to cluster or partition data, we often use Expectation-and-Maximization (EM) or Variational approximation with a Gaussian Mixture Model (GMM), which is a parametric probability density function represented as a weighted sum of $\hat{K}$ Gaussian component densities. However, model selection to find underlying $\hat{K}$ is one of the key concerns in GMM clustering, since we can obtain the desired clusters only when $\hat{K}$ is known. In this paper, we propose a new model selection algorithm to explore $\hat{K}$ in a Bayesian framework. The proposed algorithm builds the density of the model order which any information criterions such as AIC and BIC basically fail to reconstruct. In addition, this algorithm reconstructs the density quickly as compared to the time-consuming Monte Carlo simulation. △ Less

Submitted 3 July, 2013; originally announced July 2013.

A data mining approach using transaction patterns for card fraud detection

Authors: Chae Chang Lee, Ji Won yoon

Abstract: Credit and debit cards, rather than actual money, have become the universal payment means. With these cards, it has become possible to buy expensive items easily without an additional complex authentication procedure being conducted. However, card transaction features are targeted by criminals seeking to use a lost or stolen card and looking for a chance to replicate it. Accidents, whether caused… ▽ More Credit and debit cards, rather than actual money, have become the universal payment means. With these cards, it has become possible to buy expensive items easily without an additional complex authentication procedure being conducted. However, card transaction features are targeted by criminals seeking to use a lost or stolen card and looking for a chance to replicate it. Accidents, whether caused by the negligence of users or not, that lead to a transaction being performed by a criminal rather than the authorized card user should be prevented. Therefore, card companies are providing their clients with a variety of policies and standards to cover this eventuality. Card companies must therefore be able to distinguish between the rightful user and illegal users according to these standards in order to minimize damage resulting from unauthorized transactions. However, there is a limit to applying the same fixed standards to all card users, since the transaction patterns of people differ and even individuals' transaction patterns may change frequently due to changes income and consumption preference. Therefore, when only a specific threshold is applied, it is difficult to distinguish a fraudulent card transaction from a legitimate one. In this paper, we present methods for learning the individual patterns of a card user's transaction amount and the region in which he or she uses the card, for a given period, and for determining whether the specified transaction is allowable in accordance with these learned user transaction patterns. Then, we classify legitimate transactions and fraudulent transactions by setting thresholds based on the learned individual patterns. △ Less

Submitted 24 June, 2013; originally announced June 2013.

Statistical Denoising for single molecule fluorescence microscopic images

Authors: Ji Won Yoon

Abstract: Single molecule fluorescence microscopy is a powerful technique for uncovering detailed information about biological systems, both in vitro and in vivo. In such experiments, the inherently low signal to noise ratios mean that accurate algorithms to separate true signal and background noise are essential to generate meaningful results. To this end, we have developed a new and robust method to reduc… ▽ More Single molecule fluorescence microscopy is a powerful technique for uncovering detailed information about biological systems, both in vitro and in vivo. In such experiments, the inherently low signal to noise ratios mean that accurate algorithms to separate true signal and background noise are essential to generate meaningful results. To this end, we have developed a new and robust method to reduce noise in single molecule fluorescence images by using a Gaussian Markov Random Field (GMRF) prior in a Bayesian framework. Two different strategies are proposed to build the prior - an intrinsic GMRF, with a stationary relationship between pixels and a heterogeneous intrinsic GMRF, with a differently weighted relationship between pixels classified as molecules and background. Testing with synthetic and real experimental fluorescence images demonstrates that the heterogeneous intrinsic GMRF is superior to other conventional de-noising approaches. △ Less

Submitted 7 June, 2013; originally announced June 2013.

Efficient Estimation of the number of neighbours in Probabilistic K Nearest Neighbour Classification

Authors: Ji Won Yoon, Nial Friel

Abstract: Probabilistic k-nearest neighbour (PKNN) classification has been introduced to improve the performance of original k-nearest neighbour (KNN) classification algorithm by explicitly modelling uncertainty in the classification of each feature vector. However, an issue common to both KNN and PKNN is to select the optimal number of neighbours, $k$. The contribution of this paper is to incorporate the u… ▽ More Probabilistic k-nearest neighbour (PKNN) classification has been introduced to improve the performance of original k-nearest neighbour (KNN) classification algorithm by explicitly modelling uncertainty in the classification of each feature vector. However, an issue common to both KNN and PKNN is to select the optimal number of neighbours, $k$. The contribution of this paper is to incorporate the uncertainty in $k$ into the decision making, and in so doing use Bayesian model averaging to provide improved classification. Indeed the problem of assessing the uncertainty in $k$ can be viewed as one of statistical model selection which is one of the most important technical issues in the statistics and machine learning domain. In this paper, a new functional approximation algorithm is proposed to reconstruct the density of the model (order) without relying on time consuming Monte Carlo simulations. In addition, this algorithm avoids cross validation by adopting Bayesian framework. The performance of this algorithm yielded very good performance on several real experimental datasets. △ Less

Submitted 5 May, 2013; originally announced May 2013.

How to Compare the Scientific Contributions between Research Groups

Authors: Hyoungshick Kim, Ji Won Yoon

Abstract: We present a method to analyse the scientific contributions between research groups. Given multiple research groups, we construct their journal/proceeding graphs and then compute the similarity/gap between them using network analysis. This analysis can be used for measuring similarity/gap of the topics/qualities between research groups' scientific contributions. We demonstrate the practicality o… ▽ More We present a method to analyse the scientific contributions between research groups. Given multiple research groups, we construct their journal/proceeding graphs and then compute the similarity/gap between them using network analysis. This analysis can be used for measuring similarity/gap of the topics/qualities between research groups' scientific contributions. We demonstrate the practicality of our method by comparing the scientific contributions by Korean researchers with those by the global researchers for information security in 2006 - 2008. The empirical analysis shows that the current security research in South Korea has been isolated from the global research trend. △ Less

Submitted 30 October, 2009; originally announced November 2009.

Report number: 0911.0049

Hybrid Spam Filtering for Mobile Communication

Authors: Ji Won Yoon, Hyoungshick Kim, Jun Ho Huh

Abstract: Spam messages are an increasing threat to mobile communication. Several mitigation techniques have been proposed, including white and black listing, challenge-response and content-based filtering. However, none are perfect and it makes sense to use a combination rather than just one. We propose an anti-spam framework based on the hybrid of content-based filtering and challenge-response. There is… ▽ More Spam messages are an increasing threat to mobile communication. Several mitigation techniques have been proposed, including white and black listing, challenge-response and content-based filtering. However, none are perfect and it makes sense to use a combination rather than just one. We propose an anti-spam framework based on the hybrid of content-based filtering and challenge-response. There is the trade-offs between accuracy of anti-spam classifiers and the communication overhead. Experimental results show how, depending on the proportion of spam messages, different filtering %%@ parameters should be set. △ Less

Submitted 17 August, 2009; originally announced August 2009.

Comments: 6 pages, 5 figures, 1 table

ACM Class: H.4; D.2.8