-
Ring-LWE based encrypted controller with unlimited number of recursive multiplications and effect of error growth
Authors:
Yeongjun Jang,
Joowon Lee,
Seonhong Min,
Hyesun Kwak,
Junsoo Kim,
Yongsoo Song
Abstract:
In this paper, we propose a method to encrypt linear dynamic controllers that enables an unlimited number of recursive homomorphic multiplications on a Ring Learning With Errors (Ring-LWE) based cryptosystem without bootstrap**. Unlike LWE based schemes, where a scalar error is injected during encryption for security, Ring-LWE based schemes are based on polynomial rings and inject error as a pol…
▽ More
In this paper, we propose a method to encrypt linear dynamic controllers that enables an unlimited number of recursive homomorphic multiplications on a Ring Learning With Errors (Ring-LWE) based cryptosystem without bootstrap**. Unlike LWE based schemes, where a scalar error is injected during encryption for security, Ring-LWE based schemes are based on polynomial rings and inject error as a polynomial having multiple error coefficients. Such errors accumulate under recursive homomorphic operations, and it has been studied that their effect can be suppressed by the closed-loop stability when dynamic controllers are encrypted using LWE based schemes. We show that this also holds for the proposed controller encrypted using a Ring-LWE based scheme. Specifically, only the constant terms of the error polynomials affect the control performance, and their effect can be arbitrarily bounded even when the noneffective terms diverge. Furthermore, a novel packing algorithm is applied, resulting in reduced computation time and enhanced memory efficiency. Simulation results demonstrate the effectiveness of the proposed method.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
Boosted Neural Decoders: Achieving Extreme Reliability of LDPC Codes for 6G Networks
Authors:
Hee-Youl Kwak,
Dae-Young Yun,
Yongjune Kim,
Sang-Hyo Kim,
Jong-Seon No
Abstract:
Ensuring extremely high reliability is essential for channel coding in 6G networks. The next-generation of ultra-reliable and low-latency communications (xURLLC) scenario within 6G networks requires a frame error rate (FER) below 10-9. However, low-density parity-check (LDPC) codes, the standard in 5G new radio (NR), encounter a challenge known as the error floor phenomenon, which hinders to achie…
▽ More
Ensuring extremely high reliability is essential for channel coding in 6G networks. The next-generation of ultra-reliable and low-latency communications (xURLLC) scenario within 6G networks requires a frame error rate (FER) below 10-9. However, low-density parity-check (LDPC) codes, the standard in 5G new radio (NR), encounter a challenge known as the error floor phenomenon, which hinders to achieve such low rates. To tackle this problem, we introduce an innovative solution: boosted neural min-sum (NMS) decoder. This decoder operates identically to conventional NMS decoders, but is trained by novel training methods including: i) boosting learning with uncorrected vectors, ii) block-wise training schedule to address the vanishing gradient issue, iii) dynamic weight sharing to minimize the number of trainable weights, iv) transfer learning to reduce the required sample count, and v) data augmentation to expedite the sampling process. Leveraging these training strategies, the boosted NMS decoder achieves the state-of-the art performance in reducing the error floor as well as superior waterfall performance. Remarkably, we fulfill the 6G xURLLC requirement for 5G LDPC codes without the severe error floor. Additionally, the boosted NMS decoder, once its weights are trained, can perform decoding without additional modules, making it highly practical for immediate application.
△ Less
Submitted 22 May, 2024;
originally announced May 2024.
-
Topological Floquet engineering of a three-band optical lattice with dual-mode resonant driving
Authors:
Dalmin Bae,
Junyoung Park,
Myeonghyeon Kim,
Haneul Kwak,
Junhwan Kwon,
Yong-il Shin
Abstract:
We present a Floquet framework for controlling topological features of a one-dimensional optical lattice system with dual-mode resonant driving, in which both the amplitude and phase of the lattice potential are modulated simultaneously. We investigate a three-band model consisting of the three lowest orbitals and elucidate the formation of a cross-linked two-leg ladder through an indirect interba…
▽ More
We present a Floquet framework for controlling topological features of a one-dimensional optical lattice system with dual-mode resonant driving, in which both the amplitude and phase of the lattice potential are modulated simultaneously. We investigate a three-band model consisting of the three lowest orbitals and elucidate the formation of a cross-linked two-leg ladder through an indirect interband coupling via an off-resonant band. We numerically demonstrate the emergence of topologically nontrivial bands within the driven system, and a topological charge pum** phenomenon with cyclic parameter changes in the dual-mode resonant driving. Finally, we show that the band topology in the driven three-band system is protected by parity-time reversal symmetry.
△ Less
Submitted 16 May, 2024;
originally announced May 2024.
-
CrossMPT: Cross-attention Message-Passing Transformer for Error Correcting Codes
Authors:
Seong-Joon Park,
Hee-Youl Kwak,
Sang-Hyo Kim,
Yongjune Kim,
Jong-Seon No
Abstract:
Error correcting codes~(ECCs) are indispensable for reliable transmission in communication systems. The recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-attention Message-Passing Transformer~(CrossMP…
▽ More
Error correcting codes~(ECCs) are indispensable for reliable transmission in communication systems. The recent advancements in deep learning have catalyzed the exploration of ECC decoders based on neural networks. Among these, transformer-based neural decoders have achieved state-of-the-art decoding performance. In this paper, we propose a novel Cross-attention Message-Passing Transformer~(CrossMPT). CrossMPT iteratively updates two types of input vectors (i.e., magnitude and syndrome vectors) using two masked cross-attention blocks. The mask matrices in these cross-attention blocks are determined by the code's parity-check matrix that delineates the relationship between magnitude and syndrome vectors. Our experimental results show that CrossMPT significantly outperforms existing neural network-based decoders, particularly in decoding low-density parity-check codes. Notably, CrossMPT also achieves a significant reduction in computational complexity, achieving over a 50\% decrease in its attention layers compared to the original transformer-based decoder, while retaining the computational complexity of the remaining layers.
△ Less
Submitted 2 May, 2024;
originally announced May 2024.
-
Rematch: Robust and Efficient Matching of Local Knowledge Graphs to Improve Structural and Semantic Similarity
Authors:
Zoher Kachwala,
Jisun An,
Haewoon Kwak,
Filippo Menczer
Abstract:
Knowledge graphs play a pivotal role in various applications, such as question-answering and fact-checking. Abstract Meaning Representation (AMR) represents text as knowledge graphs. Evaluating the quality of these graphs involves matching them structurally to each other and semantically to the source text. Existing AMR metrics are inefficient and struggle to capture semantic similarity. We also l…
▽ More
Knowledge graphs play a pivotal role in various applications, such as question-answering and fact-checking. Abstract Meaning Representation (AMR) represents text as knowledge graphs. Evaluating the quality of these graphs involves matching them structurally to each other and semantically to the source text. Existing AMR metrics are inefficient and struggle to capture semantic similarity. We also lack a systematic evaluation benchmark for assessing structural similarity between AMR graphs. To overcome these limitations, we introduce a novel AMR similarity metric, rematch, alongside a new evaluation for structural similarity called RARE. Among state-of-the-art metrics, rematch ranks second in structural similarity; and first in semantic similarity by 1--5 percentage points on the STS-B and SICK-R benchmarks. Rematch is also five times faster than the next most efficient metric.
△ Less
Submitted 2 April, 2024;
originally announced April 2024.
-
HyperCLOVA X Technical Report
Authors:
Kang Min Yoo,
Jaegeun Han,
Sookyo In,
Heewon Jeon,
Jisu Jeong,
Jaewook Kang,
Hyunwook Kim,
Kyung-Min Kim,
Munhyong Kim,
Sungju Kim,
Donghyun Kwak,
Hanock Kwak,
Se Jung Kwon,
Bado Lee,
Dongsoo Lee,
Gichang Lee,
Jooho Lee,
Baeseong Park,
Seong** Shin,
Joonsang Yu,
Seolki Baek,
Sumin Byeon,
Eungsup Cho,
Dooseok Choe,
Jeesung Han
, et al. (371 additional authors not shown)
Abstract:
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment t…
▽ More
We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment to responsible AI. The model is evaluated across various benchmarks, including comprehensive reasoning, knowledge, commonsense, factuality, coding, math, chatting, instruction-following, and harmlessness, in both Korean and English. HyperCLOVA X exhibits strong reasoning capabilities in Korean backed by a deep understanding of the language and cultural nuances. Further analysis of the inherent bilingual nature and its extension to multilingualism highlights the model's cross-lingual proficiency and strong generalization ability to untargeted languages, including machine translation between several language pairs and cross-lingual inference tasks. We believe that HyperCLOVA X can provide helpful guidance for regions or countries in develo** their sovereign LLMs.
△ Less
Submitted 13 April, 2024; v1 submitted 2 April, 2024;
originally announced April 2024.
-
Statistical Analysis by Semiparametric Additive Regression and LSTM-FCN Based Hierarchical Classification for Computer Vision Quantification of Parkinsonian Bradykinesia
Authors:
Youngseo Cho,
In Hee Kwak,
Dohyeon Kim,
**hee Na,
Hanjoo Sung,
Jeongjae Lee,
Young Eun Kim,
Hyeo-il Ma
Abstract:
Bradykinesia, characterized by involuntary slowing or decrement of movement, is a fundamental symptom of Parkinson's Disease (PD) and is vital for its clinical diagnosis. Despite various methodologies explored to quantify bradykinesia, computer vision-based approaches have shown promising results. However, these methods often fall short in adequately addressing key bradykinesia characteristics in…
▽ More
Bradykinesia, characterized by involuntary slowing or decrement of movement, is a fundamental symptom of Parkinson's Disease (PD) and is vital for its clinical diagnosis. Despite various methodologies explored to quantify bradykinesia, computer vision-based approaches have shown promising results. However, these methods often fall short in adequately addressing key bradykinesia characteristics in repetitive limb movements: "occasional arrest" and "decrement in amplitude."
This research advances vision-based quantification of bradykinesia by introducing nuanced numerical analysis to capture decrement in amplitudes and employing a simple deep learning technique, LSTM-FCN, for precise classification of occasional arrests. Our approach structures the classification process hierarchically, tailoring it to the unique dynamics of bradykinesia in PD.
Statistical analysis of the extracted features, including those representing arrest and fatigue, has demonstrated their statistical significance in most cases. This finding underscores the importance of considering "occasional arrest" and "decrement in amplitude" in bradykinesia quantification of limb movement. Our enhanced diagnostic tool has been rigorously tested on an extensive dataset comprising 1396 motion videos from 310 PD patients, achieving an accuracy of 80.3%. The results confirm the robustness and reliability of our method.
△ Less
Submitted 31 March, 2024;
originally announced April 2024.
-
Hierarchical Climate Control Strategy for Electric Vehicles with Door-Opening Consideration
Authors:
Sanghyeon Nam,
Hye** Lee,
Youngki Kim,
Kyoung hyun Kwak,
Kyoungseok Han
Abstract:
This study proposes a novel climate control strategy for electric vehicles (EVs) by addressing door-opening interruptions, an overlooked aspect in EV thermal management. We create and validate an EV simulation model that incorporates door-opening scenarios. Three controllers are compared using the simulation model: (i) a hierarchical non-linear model predictive control (NMPC) with a unique coolant…
▽ More
This study proposes a novel climate control strategy for electric vehicles (EVs) by addressing door-opening interruptions, an overlooked aspect in EV thermal management. We create and validate an EV simulation model that incorporates door-opening scenarios. Three controllers are compared using the simulation model: (i) a hierarchical non-linear model predictive control (NMPC) with a unique coolant dividing layer and a component for cabin air inflow regulation based on door-opening signals; (ii) a single MPC controller; and (iii) a rule-based controller. The hierarchical controller outperforms, reducing door-opening temperature drops by 46.96% and 51.33% compared to single layer MPC and rule-based methods in the relevant section. Additionally, our strategy minimizes the maximum temperature gaps between the sections during recovery by 86.4% and 78.7%, surpassing single layer MPC and rule-based approaches, respectively. We believe that this result opens up future possibilities for incorporating the thermal comfort of passengers across all sections within the vehicle.
△ Less
Submitted 31 March, 2024;
originally announced April 2024.
-
ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?
Authors:
Fan Huang,
Haewoon Kwak,
Kunwoo Park,
Jisun An
Abstract:
As AI becomes more integral in our lives, the need for transparency and responsibility grows. While natural language explanations (NLEs) are vital for clarifying the reasoning behind AI decisions, evaluating them through human judgments is complex and resource-intensive due to subjectivity and the need for fine-grained ratings. This study explores the alignment between ChatGPT and human assessment…
▽ More
As AI becomes more integral in our lives, the need for transparency and responsibility grows. While natural language explanations (NLEs) are vital for clarifying the reasoning behind AI decisions, evaluating them through human judgments is complex and resource-intensive due to subjectivity and the need for fine-grained ratings. This study explores the alignment between ChatGPT and human assessments across multiple scales (i.e., binary, ternary, and 7-Likert scale). We sample 300 data instances from three NLE datasets and collect 900 human annotations for both informativeness and clarity scores as the text quality measurement. We further conduct paired comparison experiments under different ranges of subjectivity scores, where the baseline comes from 8,346 human annotations. Our results show that ChatGPT aligns better with humans in more coarse-grained scales. Also, paired comparisons and dynamic prompting (i.e., providing semantically similar examples in the prompt) improve the alignment. This research advances our understanding of large language models' capabilities to assess the text explanation quality in different configurations for responsible AI development.
△ Less
Submitted 26 March, 2024;
originally announced March 2024.
-
Benchmarking zero-shot stance detection with FlanT5-XXL: Insights from training data, prompting, and decoding strategies into its near-SoTA performance
Authors:
Rachith Aiyappa,
Shruthi Senthilmani,
Jisun An,
Haewoon Kwak,
Yong-Yeol Ahn
Abstract:
We investigate the performance of LLM-based zero-shot stance detection on tweets. Using FlanT5-XXL, an instruction-tuned open-source LLM, with the SemEval 2016 Tasks 6A, 6B, and P-Stance datasets, we study the performance and its variations under different prompts and decoding strategies, as well as the potential biases of the model. We show that the zero-shot approach can match or outperform stat…
▽ More
We investigate the performance of LLM-based zero-shot stance detection on tweets. Using FlanT5-XXL, an instruction-tuned open-source LLM, with the SemEval 2016 Tasks 6A, 6B, and P-Stance datasets, we study the performance and its variations under different prompts and decoding strategies, as well as the potential biases of the model. We show that the zero-shot approach can match or outperform state-of-the-art benchmarks, including fine-tuned models. We provide various insights into its performance including the sensitivity to instructions and prompts, the decoding strategies, the perplexity of the prompts, and to negations and oppositions present in prompts. Finally, we ensure that the LLM has not been trained on test datasets, and identify a positivity bias which may partially explain the performance differences across decoding strategie
△ Less
Submitted 29 February, 2024;
originally announced March 2024.
-
EHRNoteQA: An LLM Benchmark for Real-World Clinical Practice Using Discharge Summaries
Authors:
Sunjun Kweon,
Jiyoun Kim,
Heeyoung Kwak,
Dongchul Cha,
Hangyul Yoon,
Kwanghyun Kim,
Jeewon Yang,
Seunghyun Won,
Edward Choi
Abstract:
Discharge summaries in Electronic Health Records (EHRs) are crucial for clinical decision-making, but their length and complexity make information extraction challenging, especially when dealing with accumulated summaries across multiple patient admissions. Large Language Models (LLMs) show promise in addressing this challenge by efficiently analyzing vast and complex data. Existing benchmarks, ho…
▽ More
Discharge summaries in Electronic Health Records (EHRs) are crucial for clinical decision-making, but their length and complexity make information extraction challenging, especially when dealing with accumulated summaries across multiple patient admissions. Large Language Models (LLMs) show promise in addressing this challenge by efficiently analyzing vast and complex data. Existing benchmarks, however, fall short in properly evaluating LLMs' capabilities in this context, as they typically focus on single-note information or limited topics, failing to reflect the real-world inquiries required by clinicians. To bridge this gap, we introduce EHRNoteQA, a novel benchmark built on the MIMIC-IV EHR, comprising 962 different QA pairs each linked to distinct patients' discharge summaries. Every QA pair is initially generated using GPT-4 and then manually reviewed and refined by three clinicians to ensure clinical relevance. EHRNoteQA includes questions that require information across multiple discharge summaries and covers eight diverse topics, mirroring the complexity and diversity of real clinical inquiries. We offer EHRNoteQA in two formats: open-ended and multi-choice question answering, and propose a reliable evaluation method for each. We evaluate 27 LLMs using EHRNoteQA and examine various factors affecting the model performance (e.g., the length and number of discharge summaries). Furthermore, to validate EHRNoteQA as a reliable proxy for expert evaluations in clinical practice, we measure the correlation between the LLM performance on EHRNoteQA, and the LLM performance manually evaluated by clinicians. Results show that LLM performance on EHRNoteQA have higher correlation with clinician-evaluated performance (Spearman: 0.78, Kendall: 0.62) compared to other benchmarks, demonstrating its practical relevance in evaluating LLMs in clinical settings.
△ Less
Submitted 27 June, 2024; v1 submitted 25 February, 2024;
originally announced February 2024.
-
Token-Ensemble Text Generation: On Attacking the Automatic AI-Generated Text Detection
Authors:
Fan Huang,
Haewoon Kwak,
Jisun An
Abstract:
The robustness of AI-content detection models against cultivated attacks (e.g., paraphrasing or word switching) remains a significant concern. This study proposes a novel token-ensemble generation strategy to challenge the robustness of current AI-content detection approaches. We explore the ensemble attack strategy by completing the prompt with the next token generated from random candidate LLMs.…
▽ More
The robustness of AI-content detection models against cultivated attacks (e.g., paraphrasing or word switching) remains a significant concern. This study proposes a novel token-ensemble generation strategy to challenge the robustness of current AI-content detection approaches. We explore the ensemble attack strategy by completing the prompt with the next token generated from random candidate LLMs. We find the token-ensemble approach significantly drops the performance of AI-content detection models (The code and test sets will be released). Our findings reveal that token-ensemble generation poses a vital challenge to current detection models and underlines the need for advancing detection technologies to counter sophisticated adversarial strategies.
△ Less
Submitted 16 February, 2024;
originally announced February 2024.
-
A Distributed Inference System for Detecting Task-wise Single Trial Event-Related Potential in Stream of Satellite Images
Authors:
Sung-** Kim,
Heon-Gyu Kwak,
Hyeon-Taek Han,
Dae-Hyeok Lee,
Ji-Hoon Jeong,
Seong-Whan Lee
Abstract:
Brain-computer interface (BCI) has garnered the significant attention for their potential in various applications, with event-related potential (ERP) performing a considerable role in BCI systems. This paper introduces a novel Distributed Inference System tailored for detecting task-wise single-trial ERPs in a stream of satellite images. Unlike traditional methodologies that employ a single model…
▽ More
Brain-computer interface (BCI) has garnered the significant attention for their potential in various applications, with event-related potential (ERP) performing a considerable role in BCI systems. This paper introduces a novel Distributed Inference System tailored for detecting task-wise single-trial ERPs in a stream of satellite images. Unlike traditional methodologies that employ a single model for target detection, our system utilizes multiple models, each optimized for specific tasks, ensuring enhanced performance across varying image transition times and target onset times. Our experiments, conducted on four participants, employed two paradigms: the Normal paradigm and an AI paradigm with bounding boxes. Results indicate that our proposed system outperforms the conventional methods in both paradigms, achieving the highest $F_β$ scores. Furthermore, including bounding boxes in the AI paradigm significantly improved target recognition. This study underscores the potential of our Distributed Inference System in advancing the field of ERP detection in satellite image streams.
△ Less
Submitted 10 November, 2023;
originally announced December 2023.
-
Zero-Shot Digital Rock Image Segmentation with a Fine-Tuned Segment Anything Model
Authors:
Zhaoyang Ma,
Xupeng He,
Shuyu Sun,
Bicheng Yan,
Hyung Kwak,
Jun Gao
Abstract:
Accurate image segmentation is crucial in reservoir modelling and material characterization, enhancing oil and gas extraction efficiency through detailed reservoir models. This precision offers insights into rock properties, advancing digital rock physics understanding. However, creating pixel-level annotations for complex CT and SEM rock images is challenging due to their size and low contrast, l…
▽ More
Accurate image segmentation is crucial in reservoir modelling and material characterization, enhancing oil and gas extraction efficiency through detailed reservoir models. This precision offers insights into rock properties, advancing digital rock physics understanding. However, creating pixel-level annotations for complex CT and SEM rock images is challenging due to their size and low contrast, lengthening analysis time. This has spurred interest in advanced semi-supervised and unsupervised segmentation techniques in digital rock image analysis, promising more efficient, accurate, and less labour-intensive methods. Meta AI's Segment Anything Model (SAM) revolutionized image segmentation in 2023, offering interactive and automated segmentation with zero-shot capabilities, essential for digital rock physics with limited training data and complex image features. Despite its advanced features, SAM struggles with rock CT/SEM images due to their absence in its training set and the low-contrast nature of grayscale images. Our research fine-tunes SAM for rock CT/SEM image segmentation, optimizing parameters and handling large-scale images to improve accuracy. Experiments on rock CT and SEM images show that fine-tuning significantly enhances SAM's performance, enabling high-quality mask generation in digital rock image analysis. Our results demonstrate the feasibility and effectiveness of the fine-tuned SAM model (RockSAM) for rock images, offering segmentation without extensive training or complex labelling.
△ Less
Submitted 17 November, 2023;
originally announced November 2023.
-
Neurophysiological Response Based on Auditory Sense for Brain Modulation Using Monaural Beat
Authors:
Ha-Na Jo,
Young-Seok Kweon,
Gi-Hwan Shin,
Heon-Gyu Kwak,
Seong-Whan Lee
Abstract:
Brain modulation is a modification process of brain activity through external stimulations. However, which condition can induce the activation is still unclear. Therefore, we aimed to identify brain activation conditions using 40 Hz monaural beat (MB). Under this stimulation, auditory sense status which is determined by frequency and power range is the condition to consider. Hence, we designed fiv…
▽ More
Brain modulation is a modification process of brain activity through external stimulations. However, which condition can induce the activation is still unclear. Therefore, we aimed to identify brain activation conditions using 40 Hz monaural beat (MB). Under this stimulation, auditory sense status which is determined by frequency and power range is the condition to consider. Hence, we designed five sessions to compare; no stimulation, audible (AB), inaudible in frequency, inaudible in power, and inaudible in frequency and power. Ten healthy participants underwent each stimulation session for ten minutes with electroencephalogram (EEG) recording. For analysis, we calculated the power spectral density (PSD) of EEG for each session and compared them in frequency, time, and five brain regions. As a result, we observed the prominent power peak at 40 Hz in only AB. The induced EEG amplitude increase started at one minute and increased until the end of the session. These results of AB had significant differences in frontal, central, temporal, parietal, and occipital regions compared to other stimulations. From the statistical analysis, the PSD of the right temporal region was significantly higher than the left. We figure out the role that the auditory sense is important to lead brain activation. These findings help to understand the neurophysiological principle and effects of auditory stimulation.
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Impact of Nap on Performance in Different Working Memory Tasks Using EEG
Authors:
Gi-Hwan Shin,
Young-Seok Kweon,
Heon-Gyu Kwak,
Ha-Na Jo,
Seong-Whan Lee
Abstract:
Electroencephalography (EEG) has been widely used to study the relationship between naps and working memory, yet the effects of naps on distinct working memory tasks remain unclear. Here, participants performed word-pair and visuospatial working memory tasks pre- and post-nap sessions. We found marked differences in accuracy and reaction time between tasks performed pre- and post-nap. In order to…
▽ More
Electroencephalography (EEG) has been widely used to study the relationship between naps and working memory, yet the effects of naps on distinct working memory tasks remain unclear. Here, participants performed word-pair and visuospatial working memory tasks pre- and post-nap sessions. We found marked differences in accuracy and reaction time between tasks performed pre- and post-nap. In order to identify the impact of naps on performance in each working memory task, we employed clustering to classify participants as high- or low-performers. Analysis of sleep architecture revealed significant variations in sleep onset latency and rapid eye movement (REM) proportion. In addition, the two groups exhibited prominent differences, especially in the delta power of the Non-REM 3 stage linked to memory. Our results emphasize the interplay between nap-related neural activity and working memory, underlining specific EEG markers associated with cognitive performance.
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Influence of Video Dynamics on EEG-based Single-Trial Video Target Surveillance System
Authors:
Heon-Gyu Kwak,
Sung-** Kim,
Hyeon-Taek Han,
Ji-Hoon Jeong,
Seong-Whan Lee
Abstract:
Target detection models are one of the widely used deep learning-based applications for reducing human efforts on video surveillance and patrol. However, the application of conventional computer vision-based target detection models in military usage can result in limited performance, due to the lack of sample data of hostile targets. In this paper, we present the possibility of the electroencephal…
▽ More
Target detection models are one of the widely used deep learning-based applications for reducing human efforts on video surveillance and patrol. However, the application of conventional computer vision-based target detection models in military usage can result in limited performance, due to the lack of sample data of hostile targets. In this paper, we present the possibility of the electroencephalography-based video target detection model, which could be applied as a supportive module of the military video surveillance system. The proposed framework and detection model showed prospective performance achieving a mean macro F-beta of 0.6522 with asynchronous real-time data from five subjects, in a certain video stimulus, but not on some video stimuli. By analyzing the results of experiments using each video stimulus, we present the factors that would affect the performance of electroencephalography-based video target detection models.
△ Less
Submitted 28 February, 2024; v1 submitted 14 November, 2023;
originally announced November 2023.
-
Relationship Between Mood, Sleepiness, and EEG Functional Connectivity by 40 Hz Monaural Beats
Authors:
Ha-Na Jo,
Young-Seok Kweon,
Gi-Hwan Shin,
Heon-Gyu Kwak,
Seong-Whan Lee
Abstract:
The monaural beat is known that it can modulate brain and personal states. However, which changes in brain waves are related to changes in state is still unclear. Therefore, we aimed to investigate the effects of monaural beats and find the relationship between them. Ten participants took part in five separate random sessions, which included a baseline session and four sessions with monaural beats…
▽ More
The monaural beat is known that it can modulate brain and personal states. However, which changes in brain waves are related to changes in state is still unclear. Therefore, we aimed to investigate the effects of monaural beats and find the relationship between them. Ten participants took part in five separate random sessions, which included a baseline session and four sessions with monaural beats stimulation: one audible session and three inaudible sessions. Electroencephalogram (EEG) were recorded and participants completed pre- and post-stimulation questionnaires assessing mood and sleepiness. As a result, audible session led to increased arousal and positive mood compared to other conditions. From the neurophysiological analysis, statistical differences in frontal-central, central-central, and central-parietal connectivity were observed only in the audible session. Furthermore, a significant correlation was identified between sleepiness and EEG power in the temporal and occipital regions. These results suggested a more detailed correlation for stimulation to change its personal state. These findings have implications for applications in areas such as cognitive enhancement, mood regulation, and sleep management.
△ Less
Submitted 20 November, 2023; v1 submitted 14 November, 2023;
originally announced November 2023.
-
Multi-Signal Reconstruction Using Masked Autoencoder From EEG During Polysomnography
Authors:
Young-Seok Kweon,
Gi-Hwan Shin,
Heon-Gyu Kwak,
Ha-Na Jo,
Seong-Whan Lee
Abstract:
Polysomnography (PSG) is an indispensable diagnostic tool in sleep medicine, essential for identifying various sleep disorders. By capturing physiological signals, including EEG, EOG, EMG, and cardiorespiratory metrics, PSG presents a patient's sleep architecture. However, its dependency on complex equipment and expertise confines its use to specialized clinical settings. Addressing these limitati…
▽ More
Polysomnography (PSG) is an indispensable diagnostic tool in sleep medicine, essential for identifying various sleep disorders. By capturing physiological signals, including EEG, EOG, EMG, and cardiorespiratory metrics, PSG presents a patient's sleep architecture. However, its dependency on complex equipment and expertise confines its use to specialized clinical settings. Addressing these limitations, our study aims to perform PSG by develo** a system that requires only a single EEG measurement. We propose a novel system capable of reconstructing multi-signal PSG from a single-channel EEG based on a masked autoencoder. The masked autoencoder was trained and evaluated using the Sleep-EDF-20 dataset, with mean squared error as the metric for assessing the similarity between original and reconstructed signals. The model demonstrated proficiency in reconstructing multi-signal data. Our results present promise for the development of more accessible and long-term sleep monitoring systems. This suggests the expansion of PSG's applicability, enabling its use beyond the confines of clinics.
△ Less
Submitted 13 November, 2023;
originally announced November 2023.
-
Enhancing Rock Image Segmentation in Digital Rock Physics: A Fusion of Generative AI and State-of-the-Art Neural Networks
Authors:
Zhaoyang Ma,
Xupeng He,
Hyung Kwak,
Jun Gao,
Shuyu Sun,
Bicheng Yan
Abstract:
In digital rock physics, analysing microstructures from CT and SEM scans is crucial for estimating properties like porosity and pore connectivity. Traditional segmentation methods like thresholding and CNNs often fall short in accurately detailing rock microstructures and are prone to noise. U-Net improved segmentation accuracy but required many expert-annotated samples, a laborious and error-pron…
▽ More
In digital rock physics, analysing microstructures from CT and SEM scans is crucial for estimating properties like porosity and pore connectivity. Traditional segmentation methods like thresholding and CNNs often fall short in accurately detailing rock microstructures and are prone to noise. U-Net improved segmentation accuracy but required many expert-annotated samples, a laborious and error-prone process due to complex pore shapes. Our study employed an advanced generative AI model, the diffusion model, to overcome these limitations. This model generated a vast dataset of CT/SEM and binary segmentation pairs from a small initial dataset. We assessed the efficacy of three neural networks: U-Net, Attention-U-net, and TransUNet, for segmenting these enhanced images. The diffusion model proved to be an effective data augmentation technique, improving the generalization and robustness of deep learning models. TransU-Net, incorporating Transformer structures, demonstrated superior segmentation accuracy and IoU metrics, outperforming both U-Net and Attention-U-net. Our research advances rock image segmentation by combining the diffusion model with cutting-edge neural networks, reducing dependency on extensive expert data and boosting segmentation accuracy and robustness. TransU-Net sets a new standard in digital rock physics, paving the way for future geoscience and engineering breakthroughs.
△ Less
Submitted 10 November, 2023;
originally announced November 2023.
-
Boosting Learning for LDPC Codes to Improve the Error-Floor Performance
Authors:
Hee-Youl Kwak,
Dae-Young Yun,
Yongjune Kim,
Sang-Hyo Kim,
Jong-Seon No
Abstract:
Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios de…
▽ More
Low-density parity-check (LDPC) codes have been successfully commercialized in communication systems due to their strong error correction capabilities and simple decoding process. However, the error-floor phenomenon of LDPC codes, in which the error rate stops decreasing rapidly at a certain level, presents challenges for achieving extremely low error rates and deploying LDPC codes in scenarios demanding ultra-high reliability. In this work, we propose training methods for neural min-sum (NMS) decoders to eliminate the error-floor effect. First, by leveraging the boosting learning technique of ensemble networks, we divide the decoding network into two neural decoders and train the post decoder to be specialized for uncorrected words that the first decoder fails to correct. Secondly, to address the vanishing gradient issue in training, we introduce a block-wise training schedule that locally trains a block of weights while retraining the preceding block. Lastly, we show that assigning different weights to unsatisfied check nodes effectively lowers the error-floor with a minimal number of weights. By applying these training methods to standard LDPC codes, we achieve the best error-floor performance compared to other decoding methods. The proposed NMS decoder, optimized solely through novel training methods without additional modules, can be integrated into existing LDPC decoders without incurring extra hardware costs. The source code is available at https://github.com/ghy1228/LDPC_Error_Floor .
△ Less
Submitted 29 October, 2023; v1 submitted 11 October, 2023;
originally announced October 2023.
-
How to Mask in Error Correction Code Transformer: Systematic and Double Masking
Authors:
Seong-Joon Park,
Hee-Youl Kwak,
Sang-Hyo Kim,
Sunghwan Kim,
Yongjune Kim,
Jong-Seon No
Abstract:
In communication and storage systems, error correction codes (ECCs) are pivotal in ensuring data reliability. As deep learning's applicability has broadened across diverse domains, there is a growing research focus on neural network-based decoders that outperform traditional decoding algorithms. Among these neural decoders, Error Correction Code Transformer (ECCT) has achieved the state-of-the-art…
▽ More
In communication and storage systems, error correction codes (ECCs) are pivotal in ensuring data reliability. As deep learning's applicability has broadened across diverse domains, there is a growing research focus on neural network-based decoders that outperform traditional decoding algorithms. Among these neural decoders, Error Correction Code Transformer (ECCT) has achieved the state-of-the-art performance, outperforming other methods by large margins. To further enhance the performance of ECCT, we propose two novel methods. First, leveraging the systematic encoding technique of ECCs, we introduce a new masking matrix for ECCT, aiming to improve the performance and reduce the computational complexity. Second, we propose a novel transformer architecture of ECCT called a double-masked ECCT. This architecture employs two different mask matrices in a parallel manner to learn more diverse features of the relationship between codeword bits in the masked self-attention blocks. Extensive simulation results show that the proposed double-masked ECCT outperforms the conventional ECCT, achieving the state-of-the-art decoding performance with significant margins.
△ Less
Submitted 25 August, 2023; v1 submitted 15 August, 2023;
originally announced August 2023.
-
Evaluating the Impact of Social Determinants on Health Prediction in the Intensive Care Unit
Authors:
Ming Ying Yang,
Gloria Hyunjung Kwak,
Tom Pollard,
Leo Anthony Celi,
Marzyeh Ghassemi
Abstract:
Social determinants of health (SDOH) -- the conditions in which people live, grow, and age -- play a crucial role in a person's health and well-being. There is a large, compelling body of evidence in population health studies showing that a wide range of SDOH is strongly correlated with health outcomes. Yet, a majority of the risk prediction models based on electronic health records (EHR) do not i…
▽ More
Social determinants of health (SDOH) -- the conditions in which people live, grow, and age -- play a crucial role in a person's health and well-being. There is a large, compelling body of evidence in population health studies showing that a wide range of SDOH is strongly correlated with health outcomes. Yet, a majority of the risk prediction models based on electronic health records (EHR) do not incorporate a comprehensive set of SDOH features as they are often noisy or simply unavailable. Our work links a publicly available EHR database, MIMIC-IV, to well-documented SDOH features. We investigate the impact of such features on common EHR prediction tasks across different patient populations. We find that community-level SDOH features do not improve model performance for a general patient population, but can improve data-limited model fairness for specific subpopulations. We also demonstrate that SDOH features are vital for conducting thorough audits of algorithmic biases beyond protective attributes. We hope the new integrated EHR-SDOH database will enable studies on the relationship between community health and individual outcomes and provide new benchmarks to study algorithmic biases beyond race, gender, and age.
△ Less
Submitted 14 August, 2023; v1 submitted 21 May, 2023;
originally announced May 2023.
-
Public Perception of Generative AI on Twitter: An Empirical Study Based on Occupation and Usage
Authors:
Kunihiro Miyazaki,
Taichi Murayama,
Takayuki Uchiba,
Jisun An,
Haewoon Kwak
Abstract:
The emergence of generative AI has sparked substantial discussions, with the potential to have profound impacts on society in all aspects. As emerging technologies continue to advance, it is imperative to facilitate their proper integration into society, managing expectations and fear. This paper investigates users' perceptions of generative AI using 3M posts on Twitter from January 2019 to March…
▽ More
The emergence of generative AI has sparked substantial discussions, with the potential to have profound impacts on society in all aspects. As emerging technologies continue to advance, it is imperative to facilitate their proper integration into society, managing expectations and fear. This paper investigates users' perceptions of generative AI using 3M posts on Twitter from January 2019 to March 2023, especially focusing on their occupation and usage. We find that people across various occupations, not just IT-related ones, show a strong interest in generative AI. The sentiment toward generative AI is generally positive, and remarkably, their sentiments are positively correlated with their exposure to AI. Among occupations, illustrators show exceptionally negative sentiment mainly due to concerns about the unethical usage of artworks in constructing AI. People use ChatGPT in diverse ways, and notably the casual usage in which they "play with" ChatGPT tends to associate with positive sentiments. After the release of ChatGPT, people's interest in AI in general has increased dramatically; however, the topic with the most significant increase and positive sentiment is related to crypto, indicating the hype-worthy characteristics of generative AI. These findings would offer valuable lessons for policymaking on the emergence of new technology and also empirical insights for the considerations of future human-AI symbiosis.
△ Less
Submitted 16 May, 2023;
originally announced May 2023.
-
YouNICon: YouTube's CommuNIty of Conspiracy Videos
Authors:
Shaoyi Liaw,
Fan Huang,
Fabricio Benevenuto,
Haewoon Kwak,
Jisun An
Abstract:
Conspiracy theories are widely propagated on social media. Among various social media services, YouTube is one of the most influential sources of news and entertainment. This paper seeks to develop a dataset, YOUNICON, to enable researchers to perform conspiracy theory detection as well as classification of videos with conspiracy theories into different topics. YOUNICON is a dataset with a large c…
▽ More
Conspiracy theories are widely propagated on social media. Among various social media services, YouTube is one of the most influential sources of news and entertainment. This paper seeks to develop a dataset, YOUNICON, to enable researchers to perform conspiracy theory detection as well as classification of videos with conspiracy theories into different topics. YOUNICON is a dataset with a large collection of videos from suspicious channels that were identified to contain conspiracy theories in a previous study (Ledwich and Zaitsev 2020). Overall, YOUNICON will enable researchers to study trends in conspiracy theories and understand how individuals can interact with the conspiracy theory producing community or channel. Our data is available at: https://doi.org/10.5281/zenodo.7466262.
△ Less
Submitted 11 April, 2023;
originally announced April 2023.
-
Iterative Soft Decoding Algorithm for DNA Storage Using Quality Score and Redecoding
Authors:
Jaeho Jeong,
Hosung Park,
Hee-Youl Kwak,
Jong-Seon No,
Hahyeon Jeon,
Jeong Wook Lee,
Jae-Won Kim
Abstract:
Ever since deoxyribonucleic acid (DNA) was considered as a next-generation data-storage medium, lots of research efforts have been made to correct errors occurred during the synthesis, storage, and sequencing processes using error correcting codes (ECCs). Previous works on recovering the data from the sequenced DNA pool with errors have utilized hard decoding algorithms based on a majority decisio…
▽ More
Ever since deoxyribonucleic acid (DNA) was considered as a next-generation data-storage medium, lots of research efforts have been made to correct errors occurred during the synthesis, storage, and sequencing processes using error correcting codes (ECCs). Previous works on recovering the data from the sequenced DNA pool with errors have utilized hard decoding algorithms based on a majority decision rule. To improve the correction capability of ECCs and robustness of the DNA storage system, we propose a new iterative soft decoding algorithm, where soft information is obtained from FASTQ files and channel statistics. In particular, we propose a new formula for log-likelihood ratio (LLR) calculation using quality scores (Q-scores) and a redecoding method which may be suitable for the error correction and detection in the DNA sequencing area. Based on the widely adopted encoding scheme of the fountain code structure proposed by Erlich et al., we use three different sets of sequenced data to show consistency for the performance evaluation. The proposed soft decoding algorithm gives 2.3% ~ 7.0% improvement of the reading number reduction compared to the state-of-the-art decoding method and it is shown that it can deal with erroneous sequenced oligo reads with insertion and deletion errors.
△ Less
Submitted 7 April, 2023;
originally announced April 2023.
-
Can we trust the evaluation on ChatGPT?
Authors:
Rachith Aiyappa,
Jisun An,
Haewoon Kwak,
Yong-Yeol Ahn
Abstract:
ChatGPT, the first large language model (LLM) with mass adoption, has demonstrated remarkable performance in numerous natural language tasks. Despite its evident usefulness, evaluating ChatGPT's performance in diverse problem domains remains challenging due to the closed nature of the model and its continuous updates via Reinforcement Learning from Human Feedback (RLHF). We highlight the issue of…
▽ More
ChatGPT, the first large language model (LLM) with mass adoption, has demonstrated remarkable performance in numerous natural language tasks. Despite its evident usefulness, evaluating ChatGPT's performance in diverse problem domains remains challenging due to the closed nature of the model and its continuous updates via Reinforcement Learning from Human Feedback (RLHF). We highlight the issue of data contamination in ChatGPT evaluations, with a case study of the task of stance detection. We discuss the challenge of preventing data contamination and ensuring fair model evaluation in the age of closed and continuously trained models.
△ Less
Submitted 22 March, 2023;
originally announced March 2023.
-
Wearing Masks Implies Refuting Trump?: Towards Target-specific User Stance Prediction across Events in COVID-19 and US Election 2020
Authors:
Hong Zhang,
Haewoon Kwak,
Wei Gao,
Jisun An
Abstract:
People who share similar opinions towards controversial topics could form an echo chamber and may share similar political views toward other topics as well. The existence of such connections, which we call connected behavior, gives researchers a unique opportunity to predict how one would behave for a future event given their past behaviors. In this work, we propose a framework to conduct connecte…
▽ More
People who share similar opinions towards controversial topics could form an echo chamber and may share similar political views toward other topics as well. The existence of such connections, which we call connected behavior, gives researchers a unique opportunity to predict how one would behave for a future event given their past behaviors. In this work, we propose a framework to conduct connected behavior analysis. Neural stance detection models are trained on Twitter data collected on three seemingly independent topics, i.e., wearing a mask, racial equality, and Trump, to detect people's stance, which we consider as their online behavior in each topic-related event. Our results reveal a strong connection between the stances toward the three topical events and demonstrate the power of past behaviors in predicting one's future behavior.
△ Less
Submitted 21 March, 2023;
originally announced March 2023.
-
Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech
Authors:
Fan Huang,
Haewoon Kwak,
Jisun An
Abstract:
Recent studies have alarmed that many online hate speeches are implicit. With its subtle nature, the explainability of the detection of such hateful speech has been a challenging problem. In this work, we examine whether ChatGPT can be used for providing natural language explanations (NLEs) for implicit hateful speech detection. We design our prompt to elicit concise ChatGPT-generated NLEs and con…
▽ More
Recent studies have alarmed that many online hate speeches are implicit. With its subtle nature, the explainability of the detection of such hateful speech has been a challenging problem. In this work, we examine whether ChatGPT can be used for providing natural language explanations (NLEs) for implicit hateful speech detection. We design our prompt to elicit concise ChatGPT-generated NLEs and conduct user studies to evaluate their qualities by comparison with human-written NLEs. We discuss the potential and limitations of ChatGPT in the context of implicit hateful speech research.
△ Less
Submitted 15 March, 2023; v1 submitted 10 February, 2023;
originally announced February 2023.
-
Siamese Sleep Transformer For Robust Sleep Stage Scoring With Self-knowledge Distillation and Selective Batch Sampling
Authors:
Heon-Gyu Kwak,
Young-Seok Kweon,
Gi-Hwan Shin
Abstract:
In this paper, we propose a Siamese sleep transformer (SST) that effectively extracts features from single-channel raw electroencephalogram signals for robust sleep stage scoring. Despite the significant advances in sleep stage scoring in the last few years, most of them mainly focused on the increment of model performance. However, other problems still exist: the bias of labels in datasets and th…
▽ More
In this paper, we propose a Siamese sleep transformer (SST) that effectively extracts features from single-channel raw electroencephalogram signals for robust sleep stage scoring. Despite the significant advances in sleep stage scoring in the last few years, most of them mainly focused on the increment of model performance. However, other problems still exist: the bias of labels in datasets and the instability of model performance by repetitive training. To alleviate these problems, we propose the SST, a novel sleep stage scoring model with a selective batch sampling strategy and self-knowledge distillation. To evaluate how robust the model was to the bias of labels, we used different datasets for training and testing: the sleep heart health study and the Sleep-EDF datasets. In this condition, the SST showed competitive performance in sleep stage scoring. In addition, we demonstrated the effectiveness of the selective batch sampling strategy with a reduction of the standard deviation of performance by repetitive training. These results could show that SST extracted effective learning features against the bias of labels in datasets, and the selective batch sampling strategy worked for the model robustness in training.
△ Less
Submitted 11 December, 2022;
originally announced December 2022.
-
Development of Personalized Sleep Induction System based on Mental States
Authors:
Young-Seok Kweon,
Gi-Hwan Shin,
Heon-Gyu Kwak
Abstract:
Sleep is an essential behavior to prevent the decrement of cognitive, motor, and emotional performance and various diseases. However, it is not easy to fall asleep when people want to sleep. There are various sleep-disturbing factors such as the COVID-19 situation, noise from outside, and light during the night. We aim to develop a personalized sleep induction system based on mental states using e…
▽ More
Sleep is an essential behavior to prevent the decrement of cognitive, motor, and emotional performance and various diseases. However, it is not easy to fall asleep when people want to sleep. There are various sleep-disturbing factors such as the COVID-19 situation, noise from outside, and light during the night. We aim to develop a personalized sleep induction system based on mental states using electroencephalogram and auditory stimulation. Our system analyzes users' mental states using an electroencephalogram and results of the Pittsburgh sleep quality index and Brunel mood scale. According to mental states, the system plays sleep induction sound among five auditory stimulation: white noise, repetitive beep sounds, rainy sound, binaural beat, and sham sound. Finally, the sleep-inducing system classified the sleep stage of participants with 94.7 percent and stopped auditory stimulation if participants showed non-rapid eye movement sleep. Our system makes 18 participants fall asleep among 20 participants.
△ Less
Submitted 11 December, 2022;
originally announced December 2022.
-
Changes in Power and Information Flow in Resting-state EEG by Working Memory Process
Authors:
Gi-Hwan Shin,
Young-Seok Kweon,
Heon-Gyu Kwak
Abstract:
Many studies have analyzed working memory (WM) from electroencephalogram (EEG). However, little is known about changes in the brain neurodynamics among resting-state (RS) according to the WM process. Here, we identified frequency-specific power and information flow patterns among three RS EEG before and after WM encoding and WM retrieval. Our results demonstrated the difference in power and inform…
▽ More
Many studies have analyzed working memory (WM) from electroencephalogram (EEG). However, little is known about changes in the brain neurodynamics among resting-state (RS) according to the WM process. Here, we identified frequency-specific power and information flow patterns among three RS EEG before and after WM encoding and WM retrieval. Our results demonstrated the difference in power and information flow among RS EEG in delta (1-3.5 Hz), alpha (8-13.5 Hz), and beta (14-29.5 Hz) bands. In particular, there was a marked increase in the alpha band after WM retrieval. In addition, we calculated the association between significant characteristics of RS EEG and WM performance, and interestingly, correlations were found only in the alpha band. These results suggest that RS EEG according to the WM process has a significant impact on the variability and WM performance of brain mechanisms in relation to cognitive function.
△ Less
Submitted 11 December, 2022;
originally announced December 2022.
-
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning
Authors:
Kyuyong Shin,
Hanock Kwak,
Wonjae Kim,
Jisu Jeong,
Seungjae Jung,
Kyung-Min Kim,
Jung-Woo Ha,
Sang-Woo Lee
Abstract:
Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. Many of them benefit from utilizing users' behavior sequences as plain texts, representing rich information in any domain or system without losing generality. Hence, a question arises: Can language modeling for user history corpus help improve recommender systems? While its ver…
▽ More
Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. Many of them benefit from utilizing users' behavior sequences as plain texts, representing rich information in any domain or system without losing generality. Hence, a question arises: Can language modeling for user history corpus help improve recommender systems? While its versatile usability has been widely investigated in many domains, its applications to recommender systems still remain underexplored. We show that language modeling applied directly to task-specific user histories achieves excellent results on diverse recommendation tasks. Also, leveraging additional task-agnostic user histories delivers significant performance benefits. We further demonstrate that our approach can provide promising transfer learning capabilities for a broad spectrum of real-world recommender systems, even on unseen domains and services.
△ Less
Submitted 13 May, 2023; v1 submitted 7 December, 2022;
originally announced December 2022.
-
Political Honeymoon Effect on Social Media: Characterizing Social Media Reaction to the Changes of Prime Minister in Japan
Authors:
Kunihiro Miyazaki,
Taichi Murayama,
Akira Matsui,
Masaru Nishikawa,
Takayuki Uchiba,
Haewoon Kwak,
Jisun An
Abstract:
New leaders in democratic countries typically enjoy high approval ratings immediately after taking office. This phenomenon is called the honeymoon effect and is regarded as a significant political phenomenon; however, its mechanism remains underexplored. Therefore, this study examines how social media users respond to changes in political leadership in order to better understand the honeymoon effe…
▽ More
New leaders in democratic countries typically enjoy high approval ratings immediately after taking office. This phenomenon is called the honeymoon effect and is regarded as a significant political phenomenon; however, its mechanism remains underexplored. Therefore, this study examines how social media users respond to changes in political leadership in order to better understand the honeymoon effect in politics. In particular, we constructed a 15-year Twitter dataset on eight change timings of Japanese prime ministers consisting of 6.6M tweets and analyzed them in terms of sentiments, topics, and users. We found that, while not always, social media tend to show a honeymoon effect at the change timings of prime minister. The study also revealed that sentiment about prime ministers differed by topic, indicating that public expectations vary from one prime minister to another. Furthermore, the user base was largely replaced before and after the change in the prime minister, and their sentiment was also significantly different. The implications of this study would be beneficial for administrative management.
△ Less
Submitted 25 February, 2023; v1 submitted 6 December, 2022;
originally announced December 2022.
-
Minimum critical velocity of a Gaussian obstacle in a Bose-Einstein condensate
Authors:
Haneul Kwak,
Jong Heum Jung,
Yong-il Shin
Abstract:
When a superfluid flows past an obstacle, quantized vortices can be created in the wake above a certain critical velocity. In the experiment by Kwon et al. [Phys. Rev. A 91, 053615 (2015)], the critical velocity $v_c$ was measured for atomic Bose-Einstein condensates (BECs) using a moving repulsive Gaussian potential and $v_c$ was minimized when the potential height $V_0$ of the obstacle was close…
▽ More
When a superfluid flows past an obstacle, quantized vortices can be created in the wake above a certain critical velocity. In the experiment by Kwon et al. [Phys. Rev. A 91, 053615 (2015)], the critical velocity $v_c$ was measured for atomic Bose-Einstein condensates (BECs) using a moving repulsive Gaussian potential and $v_c$ was minimized when the potential height $V_0$ of the obstacle was close to the condensate chemical potential $μ$. Here we numerically investigate the evolution of the critical vortex shedding in a two-dimensional BEC with increasing $V_0$ and show that the minimum $v_c$ at the critical strength $V_{0c}\approx μ$ results from the local density reduction and vortex pinning effect of the repulsive obstacle. The spatial distribution of the superflow around the moving obstacle just below $v_c$ is examined. The particle density at the tip of the obstacle decreases as $V_0$ increases to $V_{c0}$ and at the critical strength, a vortex dipole is suddenly formed and dragged by the moving obstacle, indicating the onset of vortex pinning. The minimum $v_c$ exhibits power-law scaling with the obstacle size $σ$ as $v_c\sim σ^{-γ}$ with $γ\approx 1/2$.
△ Less
Submitted 13 February, 2023; v1 submitted 9 October, 2022;
originally announced October 2022.
-
Chain of Explanation: New Prompting Method to Generate Higher Quality Natural Language Explanation for Implicit Hate Speech
Authors:
Fan Huang,
Haewoon Kwak,
Jisun An
Abstract:
Recent studies have exploited advanced generative language models to generate Natural Language Explanations (NLE) for why a certain text could be hateful. We propose the Chain of Explanation (CoE) Prompting method, using the heuristic words and target group, to generate high-quality NLE for implicit hate speech. We improved the BLUE score from 44.0 to 62.3 for NLE generation by providing accurate…
▽ More
Recent studies have exploited advanced generative language models to generate Natural Language Explanations (NLE) for why a certain text could be hateful. We propose the Chain of Explanation (CoE) Prompting method, using the heuristic words and target group, to generate high-quality NLE for implicit hate speech. We improved the BLUE score from 44.0 to 62.3 for NLE generation by providing accurate target information. We then evaluate the quality of generated NLE using various automatic metrics and human annotations of informativeness and clarity scores.
△ Less
Submitted 15 March, 2023; v1 submitted 11 September, 2022;
originally announced September 2022.
-
Atomic scale evolution of the surface chemistry in Li[Ni,Mn,Co]O2 cathode for Li-ion batteries stored in air
Authors:
Mahander P. Singh,
Se-Ho Kim,
Xuyang Zhou,
Hiram Kwak,
Stoichko Antonov,
Leonardo Shoji Aota,
Chanwon Jung,
Yoon Seok Jung,
Baptiste Gault
Abstract:
Layered LiMO2 (M = Ni, Co, Mn, and Al mixture) cathode materials used for Li-ion batteries are reputed to be highly reactive through their surface, where the chemistry changes rapidly when exposed to ambient air. However, conventional electron/spectroscopy-based techniques or thermogravimetric analysis fails to capture the underlying atom-scale chemistry of vulnerable Li species. To study the evol…
▽ More
Layered LiMO2 (M = Ni, Co, Mn, and Al mixture) cathode materials used for Li-ion batteries are reputed to be highly reactive through their surface, where the chemistry changes rapidly when exposed to ambient air. However, conventional electron/spectroscopy-based techniques or thermogravimetric analysis fails to capture the underlying atom-scale chemistry of vulnerable Li species. To study the evolution of the surface composition at the atomic scale, here we use atom probe tomography and probed the surface species formed during exposure of a LiNi0.8Mn0.1Co0.1O2 (NMC811) cathode material to air. The compositional analysis evidences the formation of Li2CO3. Site specific examination from a cracked region of an NMC811 particle also suggests the predominant presence of Li2CO3. These insights will help to design improved protocols for cathode synthesis and cell assembly, as well as critical knowledge for cathode degradation
△ Less
Submitted 25 July, 2022;
originally announced July 2022.
-
You Have Earned a Trophy: Characterize In-Game Achievements and Their Completions
Authors:
Haewoon Kwak
Abstract:
Achievement systems have been actively adopted in gaming platforms to maintain players' interests. Among them, trophies in PlayStation games are one of the most successful achievement systems. While the importance of trophy design has been casually discussed in many game developers' forums, there has been no systematic study of the historical dataset of trophies yet. In this work, we construct a c…
▽ More
Achievement systems have been actively adopted in gaming platforms to maintain players' interests. Among them, trophies in PlayStation games are one of the most successful achievement systems. While the importance of trophy design has been casually discussed in many game developers' forums, there has been no systematic study of the historical dataset of trophies yet. In this work, we construct a complete dataset of PlayStation games and their trophies and investigate them from both the developers' and players' perspectives.
△ Less
Submitted 30 May, 2022;
originally announced May 2022.
-
Modeling Political Activism around Gun Debate via Social Media
Authors:
Yelena Mejova,
Jisun An,
Gianmarco De Francisci Morales,
Haewoon Kwak
Abstract:
The United States have some of the highest rates of gun violence among developed countries. Yet, there is a disagreement about the extent to which firearms should be regulated. In this study, we employ social media signals to examine the predictors of offline political activism, at both population and individual level. We show that it is possible to classify the stance of users on the gun issue, e…
▽ More
The United States have some of the highest rates of gun violence among developed countries. Yet, there is a disagreement about the extent to which firearms should be regulated. In this study, we employ social media signals to examine the predictors of offline political activism, at both population and individual level. We show that it is possible to classify the stance of users on the gun issue, especially accurately when network information is available. Alongside socioeconomic variables, network information such as the relative size of the two sides of the debate is also predictive of state-level gun policy. On individual level, we build a statistical model using network, content, and psycho-linguistic features that predicts real-life political action, and explore the most predictive linguistic features. Thus, we argue that, alongside demographics and socioeconomic indicators, social media provides useful signals in the holistic modeling of political engagement around the gun debate.
△ Less
Submitted 30 April, 2022;
originally announced May 2022.
-
Who Is Missing? Characterizing the Participation of Different Demographic Groups in a Korean Nationwide Daily Conversation Corpus
Authors:
Haewoon Kwak,
Jisun An,
Kunwoo Park
Abstract:
A conversation corpus is essential to build interactive AI applications. However, the demographic information of the participants in such corpora is largely underexplored mainly due to the lack of individual data in many corpora. In this work, we analyze a Korean nationwide daily conversation corpus constructed by the National Institute of Korean Language (NIKL) to characterize the participation o…
▽ More
A conversation corpus is essential to build interactive AI applications. However, the demographic information of the participants in such corpora is largely underexplored mainly due to the lack of individual data in many corpora. In this work, we analyze a Korean nationwide daily conversation corpus constructed by the National Institute of Korean Language (NIKL) to characterize the participation of different demographic (age and sex) groups in the corpus.
△ Less
Submitted 19 April, 2022;
originally announced April 2022.
-
Understanding Toxicity Triggers on Reddit in the Context of Singapore
Authors:
Yun Yu Chong,
Haewoon Kwak
Abstract:
While the contagious nature of online toxicity sparked increasing interest in its early detection and prevention, most of the literature focuses on the Western world. In this work, we demonstrate that 1) it is possible to detect toxicity triggers in an Asian online community, and 2) toxicity triggers can be strikingly different between Western and Eastern contexts.
While the contagious nature of online toxicity sparked increasing interest in its early detection and prevention, most of the literature focuses on the Western world. In this work, we demonstrate that 1) it is possible to detect toxicity triggers in an Asian online community, and 2) toxicity triggers can be strikingly different between Western and Eastern contexts.
△ Less
Submitted 19 April, 2022;
originally announced April 2022.
-
Characterizing Spontaneous Ideation Contest on Social Media: Case Study on the Name Change of Facebook to Meta
Authors:
Kunihiro Miyazaki,
Takayuki Uchiba,
Haewoon Kwak,
Jisun An
Abstract:
Collecting good ideas is vital for organizations, especially companies, to retain their competitiveness. Social media is gathering attention as a place to extract ideas efficiently; however, the characteristics of ideas and the posters of ideas on social media are underexamined. Thus, this study aims to characterize spontaneous ideation contests among social media users by taking an event of Faceb…
▽ More
Collecting good ideas is vital for organizations, especially companies, to retain their competitiveness. Social media is gathering attention as a place to extract ideas efficiently; however, the characteristics of ideas and the posters of ideas on social media are underexamined. Thus, this study aims to characterize spontaneous ideation contests among social media users by taking an event of Facebook's name change to Meta as a case study. As a dataset, we comprehensively collect tweets containing new acronyms of Big Tech companies, which we treat as an "idea" in this work. In the analysis, we especially focus on the diversity of ideas, which would be the main reason for enlisting social media for idea generation. As the main results, we discovered that social media users offered a wider range of ideas than those in mainstream media. The follow-follower network of the users suggested that the users' position on the network is related to the preferred ideas. Additionally, we discovered a link between the amount of user interaction on social media and the diversity of ideas. This study would promote the use of social media as a part of open innovation and co-creation processes in the industry.
△ Less
Submitted 14 November, 2022; v1 submitted 2 April, 2022;
originally announced April 2022.
-
"This is Fake News": Characterizing the Spontaneous Debunking from Twitter Users to COVID-19 False Information
Authors:
Kunihiro Miyazaki,
Takayuki Uchiba,
Kenji Tanaka,
Jisun An,
Haewoon Kwak,
Kazutoshi Sasahara
Abstract:
False information spreads on social media, and fact-checking is a potential countermeasure. However, there is a severe shortage of fact-checkers; an efficient way to scale fact-checking is desperately needed, especially in pandemics like COVID-19. In this study, we focus on spontaneous debunking by social media users, which has been missed in existing research despite its indicated usefulness for…
▽ More
False information spreads on social media, and fact-checking is a potential countermeasure. However, there is a severe shortage of fact-checkers; an efficient way to scale fact-checking is desperately needed, especially in pandemics like COVID-19. In this study, we focus on spontaneous debunking by social media users, which has been missed in existing research despite its indicated usefulness for fact-checking and countering false information. Specifically, we characterize the tweets with false information, or fake tweets, that tend to be debunked and Twitter users who often debunk fake tweets. For this analysis, we create a comprehensive dataset of responses to fake tweets, annotate a subset of them, and build a classification model for detecting debunking behaviors. We find that most fake tweets are left undebunked, spontaneous debunking is slower than other forms of responses, and spontaneous debunking exhibits partisanship in political topics. These results provide actionable insights into utilizing spontaneous debunking to scale conventional fact-checking, thereby supplementing existing research from a new perspective.
△ Less
Submitted 10 August, 2022; v1 submitted 27 March, 2022;
originally announced March 2022.
-
Scaling Law for Recommendation Models: Towards General-purpose User Representations
Authors:
Kyuyong Shin,
Hanock Kwak,
Su Young Kim,
Max Nihlen Ramstrom,
Jisu Jeong,
Jung-Woo Ha,
Kyung-Min Kim
Abstract:
Recent advancement of large-scale pretrained models such as BERT, GPT-3, CLIP, and Gopher, has shown astonishing achievements across various task domains. Unlike vision recognition and language models, studies on general-purpose user representation at scale still remain underexplored. Here we explore the possibility of general-purpose user representation learning by training a universal user encod…
▽ More
Recent advancement of large-scale pretrained models such as BERT, GPT-3, CLIP, and Gopher, has shown astonishing achievements across various task domains. Unlike vision recognition and language models, studies on general-purpose user representation at scale still remain underexplored. Here we explore the possibility of general-purpose user representation learning by training a universal user encoder at large scales. We demonstrate that the scaling law is present in user representation learning areas, where the training error scales as a power-law with the amount of computation. Our Contrastive Learning User Encoder (CLUE), optimizes task-agnostic objectives, and the resulting user embeddings stretch our expectation of what is possible to do in various downstream tasks. CLUE also shows great transferability to other domains and companies, as performances on an online experiment shows significant improvements in Click-Through-Rate (CTR). Furthermore, we also investigate how the model performance is influenced by the scale factors, such as training data size, model capacity, sequence length, and batch size. Finally, we discuss the broader impacts of CLUE in general.
△ Less
Submitted 22 November, 2022; v1 submitted 15 November, 2021;
originally announced November 2021.
-
Lost-circulation diagnostics using derivative-based type-curves for non-Newtonian mud leakage into fractured formation
Authors:
Rami Albattat,
Marwa AlSinan,
Hyung Kwak,
Hussein Hoteit
Abstract:
Drilling is a requisite operation for many industries to reach a targeted subsurface zone. Loss of circulation is a common problem that often causes interruptions to the drilling process and a reduction in efficiency. In this work, a semi-analytical solution and mud type-curves (MTC) are proposed to offer a quick and accurate diagnostic model to assess the lost-circulation of Herschel-Bulkley flui…
▽ More
Drilling is a requisite operation for many industries to reach a targeted subsurface zone. Loss of circulation is a common problem that often causes interruptions to the drilling process and a reduction in efficiency. In this work, a semi-analytical solution and mud type-curves (MTC) are proposed to offer a quick and accurate diagnostic model to assess the lost-circulation of Herschel-Bulkley fluids in fractured media. Based on the observed transient pressure and mud-loss trends, the model can estimate the effective fracture conductivity, the time-dependent cumulative mud-loss volume, and the leakage period. The behavior of lost-circulation into fractured formation can be quickly evaluated, at the drilling site, to perform useful diagnostics, such as the rate of fluid leakage, and the associated effective fracture hydraulic properties. Further, novel derivative-based mud-type-curves (DMTC) are developed to quantify the leakage of drilling fluid flow into fractures. The developed model is applied for non-Newtonian fluids exhibiting yield-power-law, including shear thickening and thinning, and Bingham plastic fluids. Proposing new dimensionless groups generates the dual type-curves, MTC and DMTC, which offer superior predictivity compared to traditional methods. Both type-curve sets are used in a dual trend matching, which significantly reduces the non-uniqueness issue that is typically encountered in type-curves. Data for lost circulation from several field cases are presented to demonstrate the applicability of the proposed method. The semi-analytical solver, combined with Monte Carlo simulations, is then applied to assess the sensitivity and uncertainty of various fluid and subsurface parameters. The proposed can serve as a quick diagnostic tool to evaluate lost-circulation in drilling operations.
△ Less
Submitted 3 November, 2021;
originally announced November 2021.
-
Predicting Anti-Asian Hateful Users on Twitter during COVID-19
Authors:
Jisun An,
Haewoon Kwak,
Claire Seungeun Lee,
Bogang Jun,
Yong-Yeol Ahn
Abstract:
We investigate predictors of anti-Asian hate among Twitter users throughout COVID-19. With the rise of xenophobia and polarization that has accompanied widespread social media usage in many nations, online hate has become a major social issue, attracting many researchers. Here, we apply natural language processing techniques to characterize social media users who began to post anti-Asian hate mess…
▽ More
We investigate predictors of anti-Asian hate among Twitter users throughout COVID-19. With the rise of xenophobia and polarization that has accompanied widespread social media usage in many nations, online hate has become a major social issue, attracting many researchers. Here, we apply natural language processing techniques to characterize social media users who began to post anti-Asian hate messages during COVID-19. We compare two user groups -- those who posted anti-Asian slurs and those who did not -- with respect to a rich set of features measured with data prior to COVID-19 and show that it is possible to predict who later publicly posted anti-Asian slurs. Our analysis of predictive features underlines the potential impact of news media and information sources that report on online hate and calls for further investigation into the role of polarized communication networks and news media.
△ Less
Submitted 15 September, 2021;
originally announced September 2021.
-
Global-Local Item Embedding for Temporal Set Prediction
Authors:
Seungjae Jung,
Young-** Park,
Jisu Jeong,
Kyung-Min Kim,
Hiun Kim,
Minkyu Kim,
Hanock Kwak
Abstract:
Temporal set prediction is becoming increasingly important as many companies employ recommender systems in their online businesses, e.g., personalized purchase prediction of shop** baskets. While most previous techniques have focused on leveraging a user's history, the study of combining it with others' histories remains untapped potential. This paper proposes Global-Local Item Embedding (GLOIE)…
▽ More
Temporal set prediction is becoming increasingly important as many companies employ recommender systems in their online businesses, e.g., personalized purchase prediction of shop** baskets. While most previous techniques have focused on leveraging a user's history, the study of combining it with others' histories remains untapped potential. This paper proposes Global-Local Item Embedding (GLOIE) that learns to utilize the temporal properties of sets across whole users as well as within a user by coining the names as global and local information to distinguish the two temporal patterns. GLOIE uses Variational Autoencoder (VAE) and dynamic graph-based model to capture global and local information and then applies attention to integrate resulting item embeddings. Additionally, we propose to use Tweedie output for the decoder of VAE as it can easily model zero-inflated and long-tailed distribution, which is more suitable for several real-world data distributions than Gaussian or multinomial counterparts. When evaluated on three public benchmarks, our algorithm consistently outperforms previous state-of-the-art methods in most ranking metrics.
△ Less
Submitted 5 September, 2021;
originally announced September 2021.
-
Construction of Protograph-Based Partially Doped Generalized LDPC Codes
Authors:
Jaewha Kim,
Jae-Won Kim,
Hee-Youl Kwak,
Jong-Seon No
Abstract:
A generalized low-density parity-check (GLDPC) code is a class of codes, where single parity check nodes in a conventional low-density parity-check (LDPC) code are replaced by linear codes with higher parity check constraints. In this paper, we introduce a new method of constructing GLDPC codes by inserting the generalized check nodes for partial do**. While the conventional protograph GLDPC cod…
▽ More
A generalized low-density parity-check (GLDPC) code is a class of codes, where single parity check nodes in a conventional low-density parity-check (LDPC) code are replaced by linear codes with higher parity check constraints. In this paper, we introduce a new method of constructing GLDPC codes by inserting the generalized check nodes for partial do**. While the conventional protograph GLDPC code dopes the protograph check nodes by replacing them with the generalized check nodes, a new GLDPC code is constructed by adding the generalized check nodes and partially do** the selected variable nodes to possess higher degrees of freedom, called a partially doped GLDPC (PD-GLDPC) code. The proposed PD-GLDPC codes can make it possible to do more accurate extrinsic information transfer (EXIT) analysis and the do** granularity can become finer in terms of the protograph than the conventional GLDPC code. We also propose the constraint for the typical minimum distance of PD-GLDPC codes and prove that the PD-GLDPC codes satisfying this condition have the linear minimum distance growth property. Furthermore, we obtain the threshold optimized protograph for both regular and irregular ensembles of the proposed PD-GLDPC codes over the binary erasure channel (BEC). Specifically, we propose the construction algorithms for both regular and irregular protograph-based PD-GLDPC codes that enable the construction of GLDPC codes with higher rates than the conventional ones. The block error rate performance of the proposed PD-GLDPC code shows that it has a reasonably good waterfall performance with low error floor and outperforms other LDPC codes for the same code rate, code length, and degree distribution.
△ Less
Submitted 5 September, 2022; v1 submitted 26 July, 2021;
originally announced July 2021.
-
One4all User Representation for Recommender Systems in E-commerce
Authors:
Kyuyong Shin,
Hanock Kwak,
Kyung-Min Kim,
Minkyu Kim,
Young-** Park,
Jisu Jeong,
Seungjae Jung
Abstract:
General-purpose representation learning through large-scale pre-training has shown promising results in the various machine learning fields. For an e-commerce domain, the objective of general-purpose, i.e., one for all, representations would be efficient applications for extensive downstream tasks such as user profiling, targeting, and recommendation tasks. In this paper, we systematically compare…
▽ More
General-purpose representation learning through large-scale pre-training has shown promising results in the various machine learning fields. For an e-commerce domain, the objective of general-purpose, i.e., one for all, representations would be efficient applications for extensive downstream tasks such as user profiling, targeting, and recommendation tasks. In this paper, we systematically compare the generalizability of two learning strategies, i.e., transfer learning through the proposed model, ShopperBERT, vs. learning from scratch. ShopperBERT learns nine pretext tasks with 79.2M parameters from 0.8B user behaviors collected over two years to produce user embeddings. As a result, the MLPs that employ our embedding method outperform more complex models trained from scratch for five out of six tasks. Specifically, the pre-trained embeddings have superiority over the task-specific supervised features and the strong baselines, which learn the auxiliary dataset for the cold-start problem. We also show the computational efficiency and embedding visualization of the pre-trained features.
△ Less
Submitted 23 May, 2021;
originally announced June 2021.
-
Irreducibly $SU(2)$-covariant quantum channels of low rank
Authors:
Euijung Chang,
Jaeyoung Kim,
Hyesun Kwak,
Hun Hee Lee,
Sang-Gyun Youn
Abstract:
We investigate information theoretic properties of low rank (less than or equal to 3) quantum channels with $SU(2)$-symmetry, where we have a complete description. We prove that PPT property coincides with entanglement-breaking property and that degradability seldomly holds in this class. In connection with these results we will demonstrate how we can compute Holevo and coherent information of tho…
▽ More
We investigate information theoretic properties of low rank (less than or equal to 3) quantum channels with $SU(2)$-symmetry, where we have a complete description. We prove that PPT property coincides with entanglement-breaking property and that degradability seldomly holds in this class. In connection with these results we will demonstrate how we can compute Holevo and coherent information of those channels. In particular, we exhibit a strong form of additivity violation of coherent information, which resembles the superactivation of coherent information of depolarizing channels.
△ Less
Submitted 3 May, 2021;
originally announced May 2021.