Showing 1–50 of 393 results for author: Ahn, S

MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention

Authors: Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, Qianhui Wu, Xufang Luo, Surin Ahn, Zhenhua Han, Amir H. Abdi, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, Lili Qiu

Abstract: The computational challenges of Large Language Model (LLM) inference remain a significant barrier to their widespread deployment, especially as prompt lengths continue to increase. Due to the quadratic complexity of the attention computation, it takes 30 minutes for an 8B LLM to process a prompt of 1M tokens (i.e., the pre-filling stage) on a single A100 GPU. Existing methods for speeding up prefi… ▽ More The computational challenges of Large Language Model (LLM) inference remain a significant barrier to their widespread deployment, especially as prompt lengths continue to increase. Due to the quadratic complexity of the attention computation, it takes 30 minutes for an 8B LLM to process a prompt of 1M tokens (i.e., the pre-filling stage) on a single A100 GPU. Existing methods for speeding up prefilling often fail to maintain acceptable accuracy or efficiency when applied to long-context LLMs. To address this gap, we introduce MInference (Milliontokens Inference), a sparse calculation method designed to accelerate pre-filling of long-sequence processing. Specifically, we identify three unique patterns in long-context attention matrices-the A-shape, Vertical-Slash, and Block-Sparsethat can be leveraged for efficient sparse computation on GPUs. We determine the optimal pattern for each attention head offline and dynamically build sparse indices based on the assigned pattern during inference. With the pattern and sparse indices, we perform efficient sparse attention calculations via our optimized GPU kernels to significantly reduce the latency in the pre-filling stage of long-context LLMs. Our proposed technique can be directly applied to existing LLMs without any modifications to the pre-training setup or additional fine-tuning. By evaluating on a wide range of downstream tasks, including InfiniteBench, RULER, PG-19, and Needle In A Haystack, and models including LLaMA-3-1M, GLM4-1M, Yi-200K, Phi-3-128K, and Qwen2-128K, we demonstrate that MInference effectively reduces inference latency by up to 10x for pre-filling on an A100, while maintaining accuracy. Our code is available at https://aka.ms/MInference. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Predicting Visual Attention in Graphic Design Documents

Authors: Souradeep Chakraborty, Zijun Wei, Conor Kelton, Seoyoung Ahn, Aruna Balasubramanian, Gregory J. Zelinsky, Dimitris Samaras

Abstract: We present a model for predicting visual attention during the free viewing of graphic design documents. While existing works on this topic have aimed at predicting static saliency of graphic designs, our work is the first attempt to predict both spatial attention and dynamic temporal order in which the document regions are fixated by gaze using a deep learning based model. We propose a two-stage m… ▽ More We present a model for predicting visual attention during the free viewing of graphic design documents. While existing works on this topic have aimed at predicting static saliency of graphic designs, our work is the first attempt to predict both spatial attention and dynamic temporal order in which the document regions are fixated by gaze using a deep learning based model. We propose a two-stage model for predicting dynamic attention on such documents, with webpages being our primary choice of document design for demonstration. In the first stage, we predict the saliency maps for each of the document components (e.g. logos, banners, texts, etc. for webpages) conditioned on the type of document layout. These component saliency maps are then jointly used to predict the overall document saliency. In the second stage, we use these layout-specific component saliency maps as the state representation for an inverse reinforcement learning model of fixation scanpath prediction during document viewing. To test our model, we collected a new dataset consisting of eye movements from 41 people freely viewing 450 webpages (the largest dataset of its kind). Experimental results show that our model outperforms existing models in both saliency and scanpath prediction for webpages, and also generalizes very well to other graphic design documents such as comics, posters, mobile UIs, etc. and natural images. △ Less

Submitted 2 July, 2024; originally announced July 2024.

Journal ref: IEEE Transactions on Multimedia 25 (2022): 4478-4493

Assessment of Clonal Hematopoiesis of Indeterminate Potential from Cardiac Magnetic Resonance Imaging using Deep Learning in a Cardio-oncology Population

Authors: Sangeon Ryu, Shawn Ahn, Jeacy Espinoza, Alokkumar Jha, Stephanie Halene, James S. Duncan, Jennifer M Kwan, Nicha C. Dvornek

Abstract: Background: We propose a novel method to identify who may likely have clonal hematopoiesis of indeterminate potential (CHIP), a condition characterized by the presence of somatic mutations in hematopoietic stem cells without detectable hematologic malignancy, using deep learning techniques. Methods: We developed a convolutional neural network (CNN) to predict CHIP status using 4 different views fr… ▽ More Background: We propose a novel method to identify who may likely have clonal hematopoiesis of indeterminate potential (CHIP), a condition characterized by the presence of somatic mutations in hematopoietic stem cells without detectable hematologic malignancy, using deep learning techniques. Methods: We developed a convolutional neural network (CNN) to predict CHIP status using 4 different views from standard delayed gadolinium-enhanced cardiac magnetic resonance imaging (CMR). We used 5-fold cross validation on 82 cardio-oncology patients to assess the performance of our model. Different algorithms were compared to find the optimal patient-level prediction method using the image-level CNN predictions. Results: We found that the best model had an area under the receiver operating characteristic curve of 0.85 and an accuracy of 82%. Conclusions: We conclude that a deep learning-based diagnostic approach for CHIP using CMR is promising. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Anisptropic plasmons in threefold Hopf semimetals

Authors: Seong** Ahn

Abstract: Threefold Hopf semimetals are a novel type of topological semimetals that possess an internal anisotropy characterized by a dipolar structure of the Berry curvature and an isotropic energy band structure consisting of a Dirac cone and a flat band. In this study, we theoretically investigate the impact of internal anisotropy on plasmons in threefold Hopf semimetals using random-phase approximation.… ▽ More Threefold Hopf semimetals are a novel type of topological semimetals that possess an internal anisotropy characterized by a dipolar structure of the Berry curvature and an isotropic energy band structure consisting of a Dirac cone and a flat band. In this study, we theoretically investigate the impact of internal anisotropy on plasmons in threefold Hopf semimetals using random-phase approximation. In contrast to the classical intuition that isotropy of the energy band dispersion leads to isotropic plasmons in the classical regime (i.e., in the wavelength limit), we find that plasmons in threefold Hopf semimetals exhibit notable anisotropy even in the long-wavelength limit. We derive an explicit analytical form of the long-wavelength plasmon frequency, and numerically demonstrate the validity of our results in a wide range of situations. Our work reveals that the anisotropy of long-wavelength plasmons can reach 25%, making it experimentally observable. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Comments: 7 pages, 5 figures

12C+12C Reaction Rates and the Evolution of a Massive Star

Authors: Gwangeon Seong, Yubin Kim, Kyu** Kwak, Sunghoon Ahn, Chaeyeon Park, Kevin Insik Hahn, Chunglee Kim

Abstract: Carbon fusion is important to understand the late stages in the evolution of a massive star. Astronomically interesting energy ranges for the 12C+12C reactions have been, however, poorly constrained by experiments. Theoretical studies on stellar evolution have relied on reaction rates that are extrapolated from those measured in higher energies. In this work, we update the carbon fusion reaction r… ▽ More Carbon fusion is important to understand the late stages in the evolution of a massive star. Astronomically interesting energy ranges for the 12C+12C reactions have been, however, poorly constrained by experiments. Theoretical studies on stellar evolution have relied on reaction rates that are extrapolated from those measured in higher energies. In this work, we update the carbon fusion reaction rates by fitting the astrophysical S-factor data obtained from direct measurements based on the Fowler, Caughlan, & Zimmerman (1975) formula. We examine the evolution of a 20 M_sun star with the updated 12C+12C reaction rates performing simulations with the MESA (Modules for Experiments for Stellar Astrophysics) code. Between 0.5 and 1 GK, the updated reaction rates are 0.35 to 0.5 times less than the rates suggested by Caughlan and Fowler (1988). The updated rates result in the increase of core temperature by about 7% and of the neutrino cooling by about a factor of three. Moreover, the carbon-burning lifetime is reduced by a factor of 2.7. The updated carbon fusion reaction rates lead to some changes in the details of the stellar evolution model, their impact seems relatively minor compared to other uncertain physical factors like convection, overshooting, rotation, and mass-loss history. The astrophysical S-factor measurements in lower energies have large errors below the Coulomb barrier. More precise measurements in lower energies for the carbon burning would be useful to improve our study and to understand the evolution of a massive star. △ Less

Submitted 19 June, 2024; originally announced June 2024.

Comments: 8 pages, 6 figures

Slot State Space Models

Authors: **dong Jiang, Fei Deng, Gautam Singh, Minseung Lee, Sung** Ahn

Abstract: Recent State Space Models (SSMs) such as S4, S5, and Mamba have shown remarkable computational benefits in long-range temporal dependency modeling. However, in many sequence modeling problems, the underlying process is inherently modular and it is of interest to have inductive biases that mimic this modular structure. In this paper, we introduce SlotSSMs, a novel framework for incorporating indepe… ▽ More Recent State Space Models (SSMs) such as S4, S5, and Mamba have shown remarkable computational benefits in long-range temporal dependency modeling. However, in many sequence modeling problems, the underlying process is inherently modular and it is of interest to have inductive biases that mimic this modular structure. In this paper, we introduce SlotSSMs, a novel framework for incorporating independent mechanisms into SSMs to preserve or encourage separation of information. Unlike conventional SSMs that maintain a monolithic state vector, SlotSSMs maintains the state as a collection of multiple vectors called slots. Crucially, the state transitions are performed independently per slot with sparse interactions across slots implemented via the bottleneck of self-attention. In experiments, we evaluate our model in object-centric video understanding, 3D visual reasoning, and video prediction tasks, which involve modeling multiple objects and their long-range temporal dependencies. We find that our proposed design offers substantial performance gains over existing sequence modeling methods. △ Less

Submitted 30 June, 2024; v1 submitted 18 June, 2024; originally announced June 2024.

Josephson Parametric Amplifier based Quantum Noise Limited Amplifier Development for Axion Search Experiments in CAPP

Authors: Sergey V. Uchaikin, **myeong Kim, Caglar Kutlu, Boris I. Ivanov, **su Kim, Arjan F. van Loo, Yasunobu Nakamura, Saebyeok Ahn, Seonjeong Oh, Minsu Ko, Yannis K. Semertzidis

Abstract: This paper provides a comprehensive overview of the development of flux-driven Josephson Parametric Amplifiers (JPAs) as Quantum Noise Limited Amplifier for axion search experiments conducted at the Center for Axion and Precision Physics Research (CAPP) of the Institute for Basic Science. It focuses on the characterization, and optimization of JPAs, which are crucial for achieving the highest sens… ▽ More This paper provides a comprehensive overview of the development of flux-driven Josephson Parametric Amplifiers (JPAs) as Quantum Noise Limited Amplifier for axion search experiments conducted at the Center for Axion and Precision Physics Research (CAPP) of the Institute for Basic Science. It focuses on the characterization, and optimization of JPAs, which are crucial for achieving the highest sensitivity in axion particle detection. We discuss various characterization techniques, methods for improving bandwidth, and the attainment of ultra-low noise temperatures. JPAs have emerged as indispensable tools in CAPPs axion search endeavors, playing a significant role in advancing our understanding of fundamental physics and unraveling the mysteries of the universe. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: 29 pages, 15 figures

Wideband tunable cavity development for axion dark matter searches using a piezoelectric motor in combination with gears

Authors: A. K. Yi, T. Seong, S. Lee, S. Ahn, B. I. Ivanov, S. V. Uchaikin, B. R. Ko, Y. K. Semertzidis

Abstract: Most search experiments sensitive to quantum chromodynamics axion dark matter benefit from microwave cavities, as electromagnetic resonators, that enhance the detected axion signal power and thus the experimental sensitivity drastically. As the possible axion mass spans multiple orders of magnitude, microwave cavities must be tunable and it is desirable for the cavity to have a tunable frequency r… ▽ More Most search experiments sensitive to quantum chromodynamics axion dark matter benefit from microwave cavities, as electromagnetic resonators, that enhance the detected axion signal power and thus the experimental sensitivity drastically. As the possible axion mass spans multiple orders of magnitude, microwave cavities must be tunable and it is desirable for the cavity to have a tunable frequency range that is as wide as possible. Since the tunable frequency range generally increases as the dimension of the conductor tuning rod increases for a given cylindrical conductor cavity system, we developed a cavity system with a large dimensional tuning rod in order to increase this. We, for the first time, employed not only a piezoelectric motor, but also gears to drive a large and accordingly heavy tuning rod, where such a combination to increase driving power can be adopted for extreme environments as is the case for axion dark matter experiments: cryogenic, high-magnetic-field, and high vacuum. Thanks to such higher power derived from the piezoelectric motor and gear combination, we realized a wideband tunable cavity whose frequency range is about 42\% of the central resonant frequency of the cavity, without sacrificing the experimental sensitivity too much. △ Less

Submitted 11 June, 2024; originally announced June 2024.

Comments: 10 pages, 7 figures

PlanDQ: Hierarchical Plan Orchestration via D-Conductor and Q-Performer

Authors: Chang Chen, Junyeob Baek, Fei Deng, Kenji Kawaguchi, Caglar Gulcehre, Sung** Ahn

Abstract: Despite the recent advancements in offline RL, no unified algorithm could achieve superior performance across a broad range of tasks. Offline \textit{value function learning}, in particular, struggles with sparse-reward, long-horizon tasks due to the difficulty of solving credit assignment and extrapolation errors that accumulates as the horizon of the task grows.~On the other hand, models that ca… ▽ More Despite the recent advancements in offline RL, no unified algorithm could achieve superior performance across a broad range of tasks. Offline \textit{value function learning}, in particular, struggles with sparse-reward, long-horizon tasks due to the difficulty of solving credit assignment and extrapolation errors that accumulates as the horizon of the task grows.~On the other hand, models that can perform well in long-horizon tasks are designed specifically for goal-conditioned tasks, which commonly perform worse than value function learning methods on short-horizon, dense-reward scenarios. To bridge this gap, we propose a hierarchical planner designed for offline RL called PlanDQ. PlanDQ incorporates a diffusion-based planner at the high level, named D-Conductor, which guides the low-level policy through sub-goals. At the low level, we used a Q-learning based approach called the Q-Performer to accomplish these sub-goals. Our experimental results suggest that PlanDQ can achieve superior or competitive performance on D4RL continuous control benchmark tasks as well as AntMaze, Kitchen, and Calvin as long-horizon tasks. △ Less

Submitted 10 June, 2024; originally announced June 2024.

FedDr+: Stabilizing Dot-regression with Global Feature Distillation for Federated Learning

Authors: Seongyoon Kim, Minchan Jeong, Sungnyun Kim, Sungwoo Cho, Sumyeong Ahn, Se-Young Yun

Abstract: Federated Learning (FL) has emerged as a pivotal framework for the development of effective global models (global FL) or personalized models (personalized FL) across clients with heterogeneous, non-iid data distribution. A key challenge in FL is client drift, where data heterogeneity impedes the aggregation of scattered knowledge. Recent studies have tackled the client drift issue by identifying s… ▽ More Federated Learning (FL) has emerged as a pivotal framework for the development of effective global models (global FL) or personalized models (personalized FL) across clients with heterogeneous, non-iid data distribution. A key challenge in FL is client drift, where data heterogeneity impedes the aggregation of scattered knowledge. Recent studies have tackled the client drift issue by identifying significant divergence in the last classifier layer. To mitigate this divergence, strategies such as freezing the classifier weights and aligning the feature extractor accordingly have proven effective. Although the local alignment between classifier and feature extractor has been studied as a crucial factor in FL, we observe that it may lead the model to overemphasize the observed classes within each client. Thus, our objectives are twofold: (1) enhancing local alignment while (2) preserving the representation of unseen class samples. This approach aims to effectively integrate knowledge from individual clients, thereby improving performance for both global and personalized FL. To achieve this, we introduce a novel algorithm named FedDr+, which empowers local model alignment using dot-regression loss. FedDr+ freezes the classifier as a simplex ETF to align the features and improves aggregated global models by employing a feature distillation mechanism to retain information about unseen/missing classes. Consequently, we provide empirical evidence demonstrating that our algorithm surpasses existing methods that use a frozen classifier to boost alignment across the diverse distribution. △ Less

Submitted 4 June, 2024; originally announced June 2024.

Pulmonary Embolism Mortality Prediction Using Multimodal Learning Based on Computed Tomography Angiography and Clinical Data

Authors: Zhusi Zhong, Helen Zhang, Fayez H. Fayad, Andrew C. Lancaster, John Sollee, Shreyas Kulkarni, Cheng Ting Lin, Jie Li, Xinbo Gao, Scott Collins, Colin Greineder, Sun H. Ahn, Harrison X. Bai, Zhicheng Jiao, Michael K. Atalay

Abstract: Purpose: Pulmonary embolism (PE) is a significant cause of mortality in the United States. The objective of this study is to implement deep learning (DL) models using Computed Tomography Pulmonary Angiography (CTPA), clinical data, and PE Severity Index (PESI) scores to predict PE mortality. Materials and Methods: 918 patients (median age 64 years, range 13-99 years, 52% female) with 3,978 CTPAs w… ▽ More Purpose: Pulmonary embolism (PE) is a significant cause of mortality in the United States. The objective of this study is to implement deep learning (DL) models using Computed Tomography Pulmonary Angiography (CTPA), clinical data, and PE Severity Index (PESI) scores to predict PE mortality. Materials and Methods: 918 patients (median age 64 years, range 13-99 years, 52% female) with 3,978 CTPAs were identified via retrospective review across three institutions. To predict survival, an AI model was used to extract disease-related imaging features from CTPAs. Imaging features and/or clinical variables were then incorporated into DL models to predict survival outcomes. Four models were developed as follows: (1) using CTPA imaging features only; (2) using clinical variables only; (3) multimodal, integrating both CTPA and clinical variables; and (4) multimodal fused with calculated PESI score. Performance and contribution from each modality were evaluated using concordance index (c-index) and Net Reclassification Improvement, respectively. Performance was compared to PESI predictions using the Wilcoxon signed-rank test. Kaplan-Meier analysis was performed to stratify patients into high- and low-risk groups. Additional factor-risk analysis was conducted to account for right ventricular (RV) dysfunction. Results: For both data sets, the PESI-fused and multimodal models achieved higher c-indices than PESI alone. Following stratification of patients into high- and low-risk groups by multimodal and PESI-fused models, mortality outcomes differed significantly (both p<0.001). A strong correlation was found between high-risk grou** and RV dysfunction. Conclusions: Multiomic DL models incorporating CTPA features, clinical data, and PESI achieved higher c-indices than PESI alone for PE survival prediction. △ Less

Submitted 5 June, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

Collective Variable Free Transition Path Sampling with Generative Flow Network

Authors: Kiyoung Seong, Seonghyun Park, Seonghwan Kim, Woo Youn Kim, Sungsoo Ahn

Abstract: Understanding transition paths between meta-stable states in molecular systems is fundamental for material design and drug discovery. However, sampling these paths via molecular dynamics simulations is computationally prohibitive due to the high-energy barriers between the meta-stable states. Recent machine learning approaches are often restricted to simple systems or rely on collective variables… ▽ More Understanding transition paths between meta-stable states in molecular systems is fundamental for material design and drug discovery. However, sampling these paths via molecular dynamics simulations is computationally prohibitive due to the high-energy barriers between the meta-stable states. Recent machine learning approaches are often restricted to simple systems or rely on collective variables (CVs) extracted from expensive domain knowledge. In this work, we propose to leverage generative flow networks (GFlowNets) to sample transition paths without relying on CVs. We reformulate the problem as amortized energy-based sampling over molecular trajectories and train a bias potential by minimizing the squared log-ratio between the target distribution and the generator, derived from the flow matching objective of GFlowNets. Our evaluation on three proteins (Alanine Dipeptide, Polyproline, and Chignolin) demonstrates that our approach, called TPS-GFN, generates more realistic and diverse transition paths than the previous CV-free machine learning approach. △ Less

Submitted 31 May, 2024; v1 submitted 30 May, 2024; originally announced May 2024.

Comments: 9 pages, 5 figures, 2 tables

Designing Prompt Analytics Dashboards to Analyze Student-ChatGPT Interactions in EFL Writing

Authors: Minsun Kim, SeonGyeom Kim, Suyoun Lee, Yoosang Yoon, Junho Myung, Haneul Yoo, Hyungseung Lim, Jieun Han, Yoonsu Kim, So-Yeon Ahn, Juho Kim, Alice Oh, Hwajung Hong, Tak Yeon Lee

Abstract: While ChatGPT has significantly impacted education by offering personalized resources for students, its integration into educational settings poses unprecedented risks, such as inaccuracies and biases in AI-generated content, plagiarism and over-reliance on AI, and privacy and security issues. To help teachers address such risks, we conducted a two-phase iterative design process that comprises sur… ▽ More While ChatGPT has significantly impacted education by offering personalized resources for students, its integration into educational settings poses unprecedented risks, such as inaccuracies and biases in AI-generated content, plagiarism and over-reliance on AI, and privacy and security issues. To help teachers address such risks, we conducted a two-phase iterative design process that comprises surveys, interviews, and prototype demonstration involving six EFL (English as a Foreign Language) teachers, who integrated ChatGPT into semester-long English essay writing classes. Based on the needs identified during the initial survey and interviews, we developed a prototype of Prompt Analytics Dashboard (PAD) that integrates the essay editing history and chat logs between students and ChatGPT. Teacher's feedback on the prototype informs additional features and unmet needs for designing future PAD, which helps them (1) analyze contextual analysis of student behaviors, (2) design an overall learning loop, and (3) develop their teaching skills. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Augmented Risk Prediction for the Onset of Alzheimer's Disease from Electronic Health Records with Large Language Models

Authors: Jiankun Wang, Sumyeong Ahn, Taykhoom Dalal, Xiaodan Zhang, Weishen Pan, Qiannan Zhang, Bin Chen, Hiroko H. Dodge, Fei Wang, Jiayu Zhou

Abstract: Alzheimer's disease (AD) is the fifth-leading cause of death among Americans aged 65 and older. Screening and early detection of AD and related dementias (ADRD) are critical for timely intervention and for identifying clinical trial participants. The widespread adoption of electronic health records (EHRs) offers an important resource for develo** ADRD screening tools such as machine learning bas… ▽ More Alzheimer's disease (AD) is the fifth-leading cause of death among Americans aged 65 and older. Screening and early detection of AD and related dementias (ADRD) are critical for timely intervention and for identifying clinical trial participants. The widespread adoption of electronic health records (EHRs) offers an important resource for develo** ADRD screening tools such as machine learning based predictive models. Recent advancements in large language models (LLMs) demonstrate their unprecedented capability of encoding knowledge and performing reasoning, which offers them strong potential for enhancing risk prediction. This paper proposes a novel pipeline that augments risk prediction by leveraging the few-shot inference power of LLMs to make predictions on cases where traditional supervised learning methods (SLs) may not excel. Specifically, we develop a collaborative pipeline that combines SLs and LLMs via a confidence-driven decision-making mechanism, leveraging the strengths of SLs in clear-cut cases and LLMs in more complex scenarios. We evaluate this pipeline using a real-world EHR data warehouse from Oregon Health \& Science University (OHSU) Hospital, encompassing EHRs from over 2.5 million patients and more than 20 million patient encounters. Our results show that our proposed approach effectively combines the power of SLs and LLMs, offering significant improvements in predictive performance. This advancement holds promise for revolutionizing ADRD screening and early detection practices, with potential implications for better strategies of patient management and thus improving healthcare. △ Less

Submitted 25 May, 2024; originally announced May 2024.

Pessimistic Backward Policy for GFlowNets

Authors: Hyosoon Jang, Yunhui Jang, Minsu Kim, **kyoo Park, Sungsoo Ahn

Abstract: This paper studies Generative Flow Networks (GFlowNets), which learn to sample objects proportionally to a given reward function through the trajectory of state transitions. In this work, we observe that GFlowNets tend to under-exploit the high-reward objects due to training on insufficient number of trajectories, which may lead to a large gap between the estimated flow and the (known) reward valu… ▽ More This paper studies Generative Flow Networks (GFlowNets), which learn to sample objects proportionally to a given reward function through the trajectory of state transitions. In this work, we observe that GFlowNets tend to under-exploit the high-reward objects due to training on insufficient number of trajectories, which may lead to a large gap between the estimated flow and the (known) reward value. In response to this challenge, we propose a pessimistic backward policy for GFlowNets (PBP-GFN), which maximizes the observed flow to align closely with the true reward for the object. We extensively evaluate PBP-GFN across eight benchmarks, including hyper-grid environment, bag generation, structured set generation, molecular generation, and four RNA sequence generation tasks. In particular, PBP-GFN enhances the discovery of high-reward objects, maintains the diversity of the objects, and consistently outperforms existing methods. △ Less

Submitted 24 May, 2024; originally announced May 2024.

Tackling Prevalent Conditions in Unsupervised Combinatorial Optimization: Cardinality, Minimum, Covering, and More

Authors: Fanchen Bu, Hyeonsoo Jo, Soo Yong Lee, Sungsoo Ahn, Kijung Shin

Abstract: Combinatorial optimization (CO) is naturally discrete, making machine learning based on differentiable optimization inapplicable. Karalias & Loukas (2020) adapted the probabilistic method to incorporate CO into differentiable optimization. Their work ignited the research on unsupervised learning for CO, composed of two main components: probabilistic objectives and derandomization. However, each co… ▽ More Combinatorial optimization (CO) is naturally discrete, making machine learning based on differentiable optimization inapplicable. Karalias & Loukas (2020) adapted the probabilistic method to incorporate CO into differentiable optimization. Their work ignited the research on unsupervised learning for CO, composed of two main components: probabilistic objectives and derandomization. However, each component confronts unique challenges. First, deriving objectives under various conditions (e.g., cardinality constraints and minimum) is nontrivial. Second, the derandomization process is underexplored, and the existing derandomization methods are either random sampling or naive rounding. In this work, we aim to tackle prevalent (i.e., commonly involved) conditions in unsupervised CO. First, we concretize the targets for objective construction and derandomization with theoretical justification. Then, for various conditions commonly involved in different CO problems, we derive nontrivial objectives and derandomization to meet the targets. Finally, we apply the derivations to various CO problems. Via extensive experiments on synthetic and real-world graphs, we validate the correctness of our derivations and show our empirical superiority w.r.t. both optimization quality and speed. △ Less

Submitted 23 May, 2024; v1 submitted 14 May, 2024; originally announced May 2024.

Comments: ICML 2024

HILCodec: High Fidelity and Lightweight Neural Audio Codec

Authors: Sunghwan Ahn, Beom Jun Woo, Min Hyun Han, Chanyeong Moon, Nam Soo Kim

Abstract: The recent advancement of end-to-end neural audio codecs enables compressing audio at very low bitrates while reconstructing the output audio with high fidelity. Nonetheless, such improvements often come at the cost of increased model complexity. In this paper, we identify and address the problems of existing neural audio codecs. We show that the performance of Wave-U-Net does not increase consist… ▽ More The recent advancement of end-to-end neural audio codecs enables compressing audio at very low bitrates while reconstructing the output audio with high fidelity. Nonetheless, such improvements often come at the cost of increased model complexity. In this paper, we identify and address the problems of existing neural audio codecs. We show that the performance of Wave-U-Net does not increase consistently as the network depth increases. We analyze the root cause of such a phenomenon and suggest a variance-constrained design. Also, we reveal various distortions in previous waveform domain discriminators and propose a novel distortion-free discriminator. The resulting model, \textit{HILCodec}, is a real-time streaming audio codec that demonstrates state-of-the-art quality across various bitrates and audio types. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Learning to Compose: Improving Object Centric Learning by Injecting Compositionality

Authors: Whie Jung, Jaehoon Yoo, Sung** Ahn, Seunghoon Hong

Abstract: Learning compositional representation is a key aspect of object-centric learning as it enables flexible systematic generalization and supports complex visual reasoning. However, most of the existing approaches rely on auto-encoding objective, while the compositionality is implicitly imposed by the architectural or algorithmic bias in the encoder. This misalignment between auto-encoding objective a… ▽ More Learning compositional representation is a key aspect of object-centric learning as it enables flexible systematic generalization and supports complex visual reasoning. However, most of the existing approaches rely on auto-encoding objective, while the compositionality is implicitly imposed by the architectural or algorithmic bias in the encoder. This misalignment between auto-encoding objective and learning compositionality often results in failure of capturing meaningful object representations. In this study, we propose a novel objective that explicitly encourages compositionality of the representations. Built upon the existing object-centric learning framework (e.g., slot attention), our method incorporates additional constraints that an arbitrary mixture of object representations from two images should be valid by maximizing the likelihood of the composite data. We demonstrate that incorporating our objective to the existing framework consistently improves the objective-centric learning and enhances the robustness to the architectural choices. △ Less

Submitted 1 May, 2024; originally announced May 2024.

GaussianTalker: Real-Time High-Fidelity Talking Head Synthesis with Audio-Driven 3D Gaussian Splatting

Authors: Kyusun Cho, Joungbin Lee, Heeji Yoon, Yeobin Hong, Jaehoon Ko, Sangjun Ahn, Seungryong Kim

Abstract: We propose GaussianTalker, a novel framework for real-time generation of pose-controllable talking heads. It leverages the fast rendering capabilities of 3D Gaussian Splatting (3DGS) while addressing the challenges of directly controlling 3DGS with speech audio. GaussianTalker constructs a canonical 3DGS representation of the head and deforms it in sync with the audio. A key insight is to encode t… ▽ More We propose GaussianTalker, a novel framework for real-time generation of pose-controllable talking heads. It leverages the fast rendering capabilities of 3D Gaussian Splatting (3DGS) while addressing the challenges of directly controlling 3DGS with speech audio. GaussianTalker constructs a canonical 3DGS representation of the head and deforms it in sync with the audio. A key insight is to encode the 3D Gaussian attributes into a shared implicit feature representation, where it is merged with audio features to manipulate each Gaussian attribute. This design exploits the spatial-aware features and enforces interactions between neighboring points. The feature embeddings are then fed to a spatial-audio attention module, which predicts frame-wise offsets for the attributes of each Gaussian. It is more stable than previous concatenation or multiplication approaches for manipulating the numerous Gaussians and their intricate parameters. Experimental results showcase GaussianTalker's superiority in facial fidelity, lip synchronization accuracy, and rendering speed compared to previous methods. Specifically, GaussianTalker achieves a remarkable rendering speed up to 120 FPS, surpassing previous benchmarks. Our code is made available at https://github.com/KU-CVLAB/GaussianTalker/ . △ Less

Submitted 25 April, 2024; v1 submitted 24 April, 2024; originally announced April 2024.

Comments: Project Page: https://ku-cvlab.github.io/GaussianTalker

Human-Machine Interaction in Automated Vehicles: Reducing Voluntary Driver Intervention

Authors: Xinzhi Zhong, Yang Zhou, Varshini Kamaraj, Zhenhao Zhou, Wissam Kontar, Dan Negrut, John D. Lee, Soyoung Ahn

Abstract: This paper develops a novel car-following control method to reduce voluntary driver interventions and improve traffic stability in Automated Vehicles (AVs). Through a combination of experimental and empirical analysis, we show how voluntary driver interventions can instigate substantial traffic disturbances that are amplified along the traffic upstream. Motivated by these findings, we present a fr… ▽ More This paper develops a novel car-following control method to reduce voluntary driver interventions and improve traffic stability in Automated Vehicles (AVs). Through a combination of experimental and empirical analysis, we show how voluntary driver interventions can instigate substantial traffic disturbances that are amplified along the traffic upstream. Motivated by these findings, we present a framework for driver intervention based on evidence accumulation (EA), which describes the evolution of the driver's distrust in automation, ultimately resulting in intervention. Informed through the EA framework, we propose a deep reinforcement learning (DRL)-based car-following control for AVs that is strategically designed to mitigate unnecessary driver intervention and improve traffic stability. Numerical experiments are conducted to demonstrate the effectiveness of the proposed control model. △ Less

Submitted 8 April, 2024; originally announced April 2024.

HyperCLOVA X Technical Report

Authors: Kang Min Yoo, Jaegeun Han, Sookyo In, Heewon Jeon, Jisu Jeong, Jaewook Kang, Hyunwook Kim, Kyung-Min Kim, Munhyong Kim, Sungju Kim, Donghyun Kwak, Hanock Kwak, Se Jung Kwon, Bado Lee, Dongsoo Lee, Gichang Lee, Jooho Lee, Baeseong Park, Seong** Shin, Joonsang Yu, Seolki Baek, Sumin Byeon, Eungsup Cho, Dooseok Choe, Jeesung Han , et al. (371 additional authors not shown)

Abstract: We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment t… ▽ More We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment to responsible AI. The model is evaluated across various benchmarks, including comprehensive reasoning, knowledge, commonsense, factuality, coding, math, chatting, instruction-following, and harmlessness, in both Korean and English. HyperCLOVA X exhibits strong reasoning capabilities in Korean backed by a deep understanding of the language and cultural nuances. Further analysis of the inherent bilingual nature and its extension to multilingualism highlights the model's cross-lingual proficiency and strong generalization ability to untargeted languages, including machine translation between several language pairs and cross-lingual inference tasks. We believe that HyperCLOVA X can provide helpful guidance for regions or countries in develo** their sovereign LLMs. △ Less

Submitted 13 April, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Comments: 44 pages; updated authors list and fixed author names

Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Prior

Authors: Jaehoon Ko, Kyusun Cho, Joungbin Lee, Heeji Yoon, Sangmin Lee, Sangjun Ahn, Seungryong Kim

Abstract: Recent methods for audio-driven talking head synthesis often optimize neural radiance fields (NeRF) on a monocular talking portrait video, leveraging its capability to render high-fidelity and 3D-consistent novel-view frames. However, they often struggle to reconstruct complete face geometry due to the absence of comprehensive 3D information in the input monocular videos. In this paper, we introdu… ▽ More Recent methods for audio-driven talking head synthesis often optimize neural radiance fields (NeRF) on a monocular talking portrait video, leveraging its capability to render high-fidelity and 3D-consistent novel-view frames. However, they often struggle to reconstruct complete face geometry due to the absence of comprehensive 3D information in the input monocular videos. In this paper, we introduce a novel audio-driven talking head synthesis framework, called Talk3D, that can faithfully reconstruct its plausible facial geometries by effectively adopting the pre-trained 3D-aware generative prior. Given the personalized 3D generative model, we present a novel audio-guided attention U-Net architecture that predicts the dynamic face variations in the NeRF space driven by audio. Furthermore, our model is further modulated by audio-unrelated conditioning tokens which effectively disentangle variations unrelated to audio features. Compared to existing methods, our method excels in generating realistic facial geometries even under extreme head poses. We also conduct extensive experiments showing our approach surpasses state-of-the-art benchmarks in terms of both quantitative and qualitative evaluations. △ Less

Submitted 29 March, 2024; originally announced March 2024.

Comments: Project page: https://ku-cvlab.github.io/Talk3D/

RECIPE4U: Student-ChatGPT Interaction Dataset in EFL Writing Education

Authors: Jieun Han, Haneul Yoo, Junho Myung, Minsun Kim, Tak Yeon Lee, So-Yeon Ahn, Alice Oh

Abstract: The integration of generative AI in education is expanding, yet empirical analyses of large-scale and real-world interactions between students and AI systems still remain limited. Addressing this gap, we present RECIPE4U (RECIPE for University), a dataset sourced from a semester-long experiment with 212 college students in English as Foreign Language (EFL) writing courses. During the study, studen… ▽ More The integration of generative AI in education is expanding, yet empirical analyses of large-scale and real-world interactions between students and AI systems still remain limited. Addressing this gap, we present RECIPE4U (RECIPE for University), a dataset sourced from a semester-long experiment with 212 college students in English as Foreign Language (EFL) writing courses. During the study, students engaged in dialogues with ChatGPT to revise their essays. RECIPE4U includes comprehensive records of these interactions, including conversation logs, students' intent, students' self-rated satisfaction, and students' essay edit histories. In particular, we annotate the students' utterances in RECIPE4U with 13 intention labels based on our coding schemes. We establish baseline results for two subtasks in task-oriented dialogue systems within educational contexts: intent detection and satisfaction estimation. As a foundational step, we explore student-ChatGPT interaction patterns through RECIPE4U and analyze them by focusing on students' dialogue, essay data statistics, and students' essay edits. We further illustrate potential applications of RECIPE4U dataset for enhancing the incorporation of LLMs in educational frameworks. RECIPE4U is publicly available at https://zeunie.github.io/RECIPE4U/. △ Less

Submitted 13 March, 2024; originally announced March 2024.

Comments: arXiv admin note: text overlap with arXiv:2309.13243

UFO: Uncertainty-aware LiDAR-image Fusion for Off-road Semantic Terrain Map Estimation

Authors: Ohn Kim, Junwon Seo, Seongyong Ahn, Chong Hui Kim

Abstract: Autonomous off-road navigation requires an accurate semantic understanding of the environment, often converted into a bird's-eye view (BEV) representation for various downstream tasks. While learning-based methods have shown success in generating local semantic terrain maps directly from sensor data, their efficacy in off-road environments is hindered by challenges in accurately representing uncer… ▽ More Autonomous off-road navigation requires an accurate semantic understanding of the environment, often converted into a bird's-eye view (BEV) representation for various downstream tasks. While learning-based methods have shown success in generating local semantic terrain maps directly from sensor data, their efficacy in off-road environments is hindered by challenges in accurately representing uncertain terrain features. This paper presents a learning-based fusion method for generating dense terrain classification maps in BEV. By performing LiDAR-image fusion at multiple scales, our approach enhances the accuracy of semantic maps generated from an RGB image and a single-sweep LiDAR scan. Utilizing uncertainty-aware pseudo-labels further enhances the network's ability to learn reliably in off-road environments without requiring precise 3D annotations. By conducting thorough experiments using off-road driving datasets, we demonstrate that our method can improve accuracy in off-road terrains, validating its efficacy in facilitating reliable and safe autonomous navigation in challenging off-road settings. △ Less

Submitted 4 March, 2024; originally announced March 2024.

Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming

Authors: Hany Hamed, Subin Kim, Dongyeong Kim, Jaesik Yoon, Sung** Ahn

Abstract: Model-based reinforcement learning (MBRL) has been a primary approach to ameliorating the sample efficiency issue as well as to make a generalist agent. However, there has not been much effort toward enhancing the strategy of dreaming itself. Therefore, it is a question whether and how an agent can "dream better" in a more structured and strategic way. In this paper, inspired by the observation fr… ▽ More Model-based reinforcement learning (MBRL) has been a primary approach to ameliorating the sample efficiency issue as well as to make a generalist agent. However, there has not been much effort toward enhancing the strategy of dreaming itself. Therefore, it is a question whether and how an agent can "dream better" in a more structured and strategic way. In this paper, inspired by the observation from cognitive science suggesting that humans use a spatial divide-and-conquer strategy in planning, we propose a new MBRL agent, called Dr. Strategy, which is equipped with a novel Dreaming Strategy. The proposed agent realizes a version of divide-and-conquer-like strategy in dreaming. This is achieved by learning a set of latent landmarks and then utilizing these to learn a landmark-conditioned highway policy. With the highway policy, the agent can first learn in the dream to move to a landmark, and from there it tackles the exploration and achievement task in a more focused way. In experiments, we show that the proposed model outperforms prior pixel-based MBRL methods in various visually complex and partially observable navigation tasks. △ Less

Submitted 4 June, 2024; v1 submitted 29 February, 2024; originally announced February 2024.

Comments: First two authors contributed equally

Parallelized Spatiotemporal Binding

Authors: Gautam Singh, Yue Wang, Jiawei Yang, Boris Ivanovic, Sung** Ahn, Marco Pavone, Tong Che

Abstract: While modern best practices advocate for scalable architectures that support long-range interactions, object-centric models are yet to fully embrace these architectures. In particular, existing object-centric models for handling sequential inputs, due to their reliance on RNN-based implementation, show poor stability and capacity and are slow to train on long sequences. We introduce Parallelizable… ▽ More While modern best practices advocate for scalable architectures that support long-range interactions, object-centric models are yet to fully embrace these architectures. In particular, existing object-centric models for handling sequential inputs, due to their reliance on RNN-based implementation, show poor stability and capacity and are slow to train on long sequences. We introduce Parallelizable Spatiotemporal Binder or PSB, the first temporally-parallelizable slot learning architecture for sequential inputs. Unlike conventional RNN-based approaches, PSB produces object-centric representations, known as slots, for all time-steps in parallel. This is achieved by refining the initial slots across all time-steps through a fixed number of layers equipped with causal attention. By capitalizing on the parallelism induced by our architecture, the proposed model exhibits a significant boost in efficiency. In experiments, we test PSB extensively as an encoder within an auto-encoding framework paired with a wide variety of decoder options. Compared to the state-of-the-art, our architecture demonstrates stable training on longer sequences, achieves parallelization that results in a 60% increase in training speed, and yields performance that is on par with or better on unsupervised 2D and 3D object-centric scene decomposition and understanding. △ Less

Submitted 26 February, 2024; originally announced February 2024.

Comments: See project page at https://parallel-st-binder.github.io

DREsS: Dataset for Rubric-based Essay Scoring on EFL Writing

Authors: Haneul Yoo, Jieun Han, So-Yeon Ahn, Alice Oh

Abstract: Automated essay scoring (AES) is a useful tool in English as a Foreign Language (EFL) writing education, offering real-time essay scores for students and instructors. However, previous AES models were trained on essays and scores irrelevant to the practical scenarios of EFL writing education and usually provided a single holistic score due to the lack of appropriate datasets. In this paper, we rel… ▽ More Automated essay scoring (AES) is a useful tool in English as a Foreign Language (EFL) writing education, offering real-time essay scores for students and instructors. However, previous AES models were trained on essays and scores irrelevant to the practical scenarios of EFL writing education and usually provided a single holistic score due to the lack of appropriate datasets. In this paper, we release DREsS, a large-scale, standard dataset for rubric-based automated essay scoring. DREsS comprises three sub-datasets: DREsS_New, DREsS_Std., and DREsS_CASE. We collect DREsS_New, a real-classroom dataset with 1.7K essays authored by EFL undergraduate students and scored by English education experts. We also standardize existing rubric-based essay scoring datasets as DREsS_Std. We suggest CASE, a corruption-based augmentation strategy for essays, which generates 20K synthetic samples of DREsS_CASE and improves the baseline results by 45.44%. DREsS will enable further research to provide a more accurate and practical AES system for EFL writing education. △ Less

Submitted 21 February, 2024; originally announced February 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2310.05191

Cluster structure of 3$α$+p states in $^{13}$N

Authors: J. Bishop, G. V. Rogachev, S. Ahn, M. Barbui, S. M. Cha, E. Harris, C. Hunt, C. H. Kim, D. Kim, S. H. Kim, E. Koshchiy, Z. Luo, C. Park, C. E. Parker, E. C. Pollacco, B. T. Roeder, M. Roosa, A. Saastamoinen, D. P. Scriven

Abstract: Background: Cluster states in $^{13}$N are extremely difficult to measure due to the unavailability of $^{9}$B+$α$ elastic scattering data. Purpose: Using $β$-delayed charged-particle spectroscopy of $^{13}$O, clustered states in $^{13}$N can be populated and measured in the 3$α$+p decay channel. Method: One-at-a-time implantation/decay of $^{13}$O was performed with the Texas Active Target Time P… ▽ More Background: Cluster states in $^{13}$N are extremely difficult to measure due to the unavailability of $^{9}$B+$α$ elastic scattering data. Purpose: Using $β$-delayed charged-particle spectroscopy of $^{13}$O, clustered states in $^{13}$N can be populated and measured in the 3$α$+p decay channel. Method: One-at-a-time implantation/decay of $^{13}$O was performed with the Texas Active Target Time Projection Chamber (TexAT TPC). 149 $β3αp$ decay events were observed and the excitation function in $^{13}$N reconstructed. Results: Four previously unknown $α$-decaying excited states were observed in $^{13}$N at an excitation energy of 11.3 MeV, 12.4 MeV, 13.1 MeV and 13.7 MeV decaying via the 3$α$+p channel. Conclusion: These states are seen to have a [$^{9}\mathrm{B}(\mathrm{g.s}) \bigotimes α$/ $p+^{12}\mathrm{C}(0_{2}^{+})$], [$^{9}\mathrm{B}(\frac{1}{2}^{+}) \bigotimes α$], [$^{9}\mathrm{B}(\frac{5}{2}^{+}) \bigotimes α$] and [$^{9}\mathrm{B}(\frac{5}{2}^{+}) \bigotimes α$] structure respectively. A previously-seen state at 11.8 MeV was also determined to have a [$p+^{12}\mathrm{C}(\mathrm{g.s.})$/ $p+^{12}\mathrm{C}(0_{2}^{+})$] structure. The overall magnitude of the clustering is not able to be extracted however due to the lack of a total width measurement. Clustered states in $^{13}$N (with unknown magnitude) seem to persist from the addition of a proton to the highly $α$-clustered $^{12}$C. Evidence of the $\frac{1}{2}^{+}$ state in $^{9}$B was also seen to be populated by decays from $^{13}$N$^{\star}$. △ Less

Submitted 26 February, 2024; originally announced February 2024.

Comments: arXiv admin note: substantial text overlap with arXiv:2302.14111

Spatially-Aware Transformer for Embodied Agents

Authors: Junmo Cho, Jaesik Yoon, Sung** Ahn

Abstract: Episodic memory plays a crucial role in various cognitive processes, such as the ability to mentally recall past events. While cognitive science emphasizes the significance of spatial context in the formation and retrieval of episodic memory, the current primary approach to implementing episodic memory in AI systems is through transformers that store temporally ordered experiences, which overlooks… ▽ More Episodic memory plays a crucial role in various cognitive processes, such as the ability to mentally recall past events. While cognitive science emphasizes the significance of spatial context in the formation and retrieval of episodic memory, the current primary approach to implementing episodic memory in AI systems is through transformers that store temporally ordered experiences, which overlooks the spatial dimension. As a result, it is unclear how the underlying structure could be extended to incorporate the spatial axis beyond temporal order alone and thereby what benefits can be obtained. To address this, this paper explores the use of Spatially-Aware Transformer models that incorporate spatial information. These models enable the creation of place-centric episodic memory that considers both temporal and spatial dimensions. Adopting this approach, we demonstrate that memory utilization efficiency can be improved, leading to enhanced accuracy in various place-centric downstream tasks. Additionally, we propose the Adaptive Memory Allocator, a memory management method based on reinforcement learning that aims to optimize efficiency of memory utilization. Our experiments demonstrate the advantages of our proposed model in various environments and across multiple downstream tasks, including prediction, generation, reasoning, and reinforcement learning. The source code for our models and experiments will be available at https://github.com/junmokane/spatially-aware-transformer. △ Less

Submitted 29 February, 2024; v1 submitted 23 February, 2024; originally announced February 2024.

Comments: ICLR 2024 Spotlight. First two authors contributed equally

Extensive search for axion dark matter over 1\,GHz with CAPP's Main Axion eXperiment

Authors: Saebyeok Ahn, **Myeong Kim, Boris I. Ivanov, Ohjoon Kwon, HeeSu Byun, Arjan F. van Loo, SeongTae Par, Junu Jeong, Soohyung Lee, **su Kim, Çağlar Kutlu, Andrew K. Yi, Yasunobu Nakamura, Seonjeong Oh, Danho Ahn, SungJae Bae, Hyoungsoon Choi, Jihoon Choi, Yonuk Chong, Woohyun Chung, Violeta Gkika, Jihn E. Kim, Younggeun Kim, Byeong Rok Ko, Lino Miceli , et al. (11 additional authors not shown)

Abstract: We report an extensive high-sensitivity search for axion dark matter above 1\,GHz at the Center for Axion and Precision Physics Research (CAPP). The cavity resonant search, exploiting the coupling between axions and photons, explored the frequency (mass) range of 1.025\,GHz (4.24\,$μ$eV) to 1.185\,GHz (4.91\,$μ$eV). We have introduced a number of innovations in this field, demonstrating the practi… ▽ More We report an extensive high-sensitivity search for axion dark matter above 1\,GHz at the Center for Axion and Precision Physics Research (CAPP). The cavity resonant search, exploiting the coupling between axions and photons, explored the frequency (mass) range of 1.025\,GHz (4.24\,$μ$eV) to 1.185\,GHz (4.91\,$μ$eV). We have introduced a number of innovations in this field, demonstrating the practical approach of optimizing all the relevant parameters of axion haloscopes, extending presently available technology. The CAPP 12\,T magnet with an aperture of 320\,mm made of Nb$_3$Sn and NbTi superconductors surrounding a 37-liter ultralight-weight copper cavity is expected to convert DFSZ axions into approximately $10^2$ microwave photons per second. A powerful dilution refrigerator, capable of kee** the core system below 40\,mK, combined with quantum-noise limited readout electronics, achieved a total system noise of about 200\,mK or below, which corresponds to a background of roughly $4\times 10^3$ photons per second within the axion bandwidth. The combination of all those improvements provides unprecedented search performance, imposing the most stringent exclusion limits on axion--photon coupling in this frequency range to date. These results also suggest an experimental capability suitable for highly-sensitive searches for axion dark matter above 1\,GHz. △ Less

Submitted 20 February, 2024; originally announced February 2024.

Comments: A detailed axion dark matter article with 27 pages, 22 figures

Dynamic and Super-Personalized Media Ecosystem Driven by Generative AI: Unpredictable Plays Never Repeating The Same

Authors: Sungjun Ahn, Hyun-Jeong Yim, Youngwan Lee, Sung-Ik Park

Abstract: This paper introduces a media service model that exploits artificial intelligence (AI) video generators at the receive end. This proposal deviates from the traditional multimedia ecosystem, completely relying on in-house production, by shifting part of the content creation onto the receiver. We bring a semantic process into the framework, allowing the distribution network to provide service elemen… ▽ More This paper introduces a media service model that exploits artificial intelligence (AI) video generators at the receive end. This proposal deviates from the traditional multimedia ecosystem, completely relying on in-house production, by shifting part of the content creation onto the receiver. We bring a semantic process into the framework, allowing the distribution network to provide service elements that prompt the content generator, rather than distributing encoded data of fully finished programs. The service elements include fine-tailored text descriptions, lightweight image data of some objects, or application programming interfaces, comprehensively referred to as semantic sources, and the user terminal translates the received semantic data into video frames. Empowered by the random nature of generative AI, the users could then experience super-personalized services accordingly. The proposed idea incorporates the situations in which the user receives different service providers' element packages; a sequence of packages over time, or multiple packages at the same time. Given promised in-context coherence and content integrity, the combinatory dynamics will amplify the service diversity, allowing the users to always chance upon new experiences. This work particularly aims at short-form videos and advertisements, which the users would easily feel fatigued by seeing the same frame sequence every time. In those use cases, the content provider's role will be recast as scripting semantic sources, transformed from a thorough producer. Overall, this work explores a new form of media ecosystem facilitated by receiver-embedded generative models, featuring both random content dynamics and enhanced delivery efficiency simultaneously. △ Less

Submitted 18 February, 2024; originally announced February 2024.

Comments: 13 pages, 7 figures

Decoupled Sequence and Structure Generation for Realistic Antibody Design

Authors: Nayoung Kim, Minsu Kim, Sungsoo Ahn, **kyoo Park

Abstract: Antibody design plays a pivotal role in advancing therapeutics. Although deep learning has made rapid progress in this field, existing methods jointly generate antibody sequences and structures, limiting task-specific optimization. In response, we propose an antibody sequence-structure decoupling (ASSD) framework, which separates sequence generation and structure prediction. Although our approach… ▽ More Antibody design plays a pivotal role in advancing therapeutics. Although deep learning has made rapid progress in this field, existing methods jointly generate antibody sequences and structures, limiting task-specific optimization. In response, we propose an antibody sequence-structure decoupling (ASSD) framework, which separates sequence generation and structure prediction. Although our approach is simple, such a decoupling strategy has been overlooked in previous works. We also find that the widely used non-autoregressive generators promote sequences with overly repeating tokens. Such sequences are both out-of-distribution and prone to undesirable developability properties that can trigger harmful immune responses in patients. To resolve this, we introduce a composition-based objective that allows an efficient trade-off between high performance and low token repetition. Our results demonstrate that ASSD consistently outperforms existing antibody design models, while the composition-based objective successfully mitigates token repetition of non-autoregressive models. Our code is available at \url{https://github.com/lkny123/ASSD_public}. △ Less

Submitted 27 May, 2024; v1 submitted 8 February, 2024; originally announced February 2024.

Comments: 18 pages, 6 figures

Hybrid Neural Representations for Spherical Data

Authors: Hyomin Kim, Yunhui Jang, Jaeho Lee, Sungsoo Ahn

Abstract: In this paper, we study hybrid neural representations for spherical data, a domain of increasing relevance in scientific research. In particular, our work focuses on weather and climate data as well as comic microwave background (CMB) data. Although previous studies have delved into coordinate-based neural representations for spherical signals, they often fail to capture the intricate details of h… ▽ More In this paper, we study hybrid neural representations for spherical data, a domain of increasing relevance in scientific research. In particular, our work focuses on weather and climate data as well as comic microwave background (CMB) data. Although previous studies have delved into coordinate-based neural representations for spherical signals, they often fail to capture the intricate details of highly nonlinear signals. To address this limitation, we introduce a novel approach named Hybrid Neural Representations for Spherical data (HNeR-S). Our main idea is to use spherical feature-grids to obtain positional features which are combined with a multilayer perception to predict the target signal. We consider feature-grids with equirectangular and hierarchical equal area isolatitude pixelization structures that align with weather data and CMB data, respectively. We extensively verify the effectiveness of our HNeR-S for regression, super-resolution, temporal interpolation, and compression tasks. △ Less

Submitted 5 February, 2024; originally announced February 2024.

Comments: 13 pages, 8 figures

Gaussian Plane-Wave Neural Operator for Electron Density Estimation

Authors: Seongsu Kim, Sungsoo Ahn

Abstract: This work studies machine learning for electron density prediction, which is fundamental for understanding chemical systems and density functional theory (DFT) simulations. To this end, we introduce the Gaussian plane-wave neural operator (GPWNO), which operates in the infinite-dimensional functional space using the plane-wave and Gaussian-type orbital bases, widely recognized in the context of DF… ▽ More This work studies machine learning for electron density prediction, which is fundamental for understanding chemical systems and density functional theory (DFT) simulations. To this end, we introduce the Gaussian plane-wave neural operator (GPWNO), which operates in the infinite-dimensional functional space using the plane-wave and Gaussian-type orbital bases, widely recognized in the context of DFT. In particular, both high- and low-frequency components of the density can be effectively represented due to the complementary nature of the two bases. Extensive experiments on QM9, MD, and material project datasets demonstrate GPWNO's superior performance over ten baselines. △ Less

Submitted 13 June, 2024; v1 submitted 5 February, 2024; originally announced February 2024.

Comments: Accepted by ICML 2024 main poster

Journal ref: International Conference on Machine Learning (ICML), 2024

Neural Language of Thought Models

Authors: Yi-Fu Wu, Minseung Lee, Sung** Ahn

Abstract: The Language of Thought Hypothesis suggests that human cognition operates on a structured, language-like system of mental representations. While neural language models can naturally benefit from the compositional structure inherently and explicitly expressed in language data, learning such representations from non-linguistic general observations, like images, remains a challenge. In this work, we… ▽ More The Language of Thought Hypothesis suggests that human cognition operates on a structured, language-like system of mental representations. While neural language models can naturally benefit from the compositional structure inherently and explicitly expressed in language data, learning such representations from non-linguistic general observations, like images, remains a challenge. In this work, we introduce the Neural Language of Thought Model (NLoTM), a novel approach for unsupervised learning of LoTH-inspired representation and generation. NLoTM comprises two key components: (1) the Semantic Vector-Quantized Variational Autoencoder, which learns hierarchical, composable discrete representations aligned with objects and their properties, and (2) the Autoregressive LoT Prior, an autoregressive transformer that learns to generate semantic concept tokens compositionally, capturing the underlying data distribution. We evaluate NLoTM on several 2D and 3D image datasets, demonstrating superior performance in downstream tasks, out-of-distribution generalization, and image generation quality compared to patch-based VQ-VAE and continuous object-centric representations. Our work presents a significant step towards creating neural networks exhibiting more human-like understanding by develo** LoT-like representations and offers insights into the intersection of cognitive science and machine learning. △ Less

Submitted 16 April, 2024; v1 submitted 2 February, 2024; originally announced February 2024.

Comments: Accepted in ICLR 2024

Truck Parking Usage Prediction with Decomposed Graph Neural Networks

Authors: Rei Tamaru, Yang Cheng, Steven Parker, Ernie Perry, Bin Ran, Soyoung Ahn

Abstract: Truck parking on freight corridors faces various challenges, such as insufficient parking spaces and compliance with Hour-of-Service (HOS) regulations. These constraints often result in unauthorized parking practices, causing safety concerns. To enhance the safety of freight operations, providing accurate parking usage prediction proves to be a cost-effective solution. Despite the existing researc… ▽ More Truck parking on freight corridors faces various challenges, such as insufficient parking spaces and compliance with Hour-of-Service (HOS) regulations. These constraints often result in unauthorized parking practices, causing safety concerns. To enhance the safety of freight operations, providing accurate parking usage prediction proves to be a cost-effective solution. Despite the existing research demonstrating satisfactory accuracy for predicting individual truck parking site usage, few approaches have been proposed for predicting usage with spatial dependencies of multiple truck parking sites. We present the Regional Temporal Graph Neural Network (RegT-GCN) as a predictive framework for assessing parking usage across the entire state to provide better truck parking information and mitigate unauthorized parking. The framework leverages the topological structures of truck parking site distributions and historical parking data to predict occupancy rates across a state. To achieve this, we introduce a Regional Decomposition approach, which effectively captures the geographical characteristics. We also introduce the spatial module working efficiently with the temporal module. Evaluation results demonstrate that the proposed model surpasses other baseline models, improving the performance by more than $20\%$ compared with the original model. The proposed model allows truck parking sites' percipience of the topological structures and provides higher performance. △ Less

Submitted 23 January, 2024; originally announced January 2024.

Comments: 10 pages, 5 figures, 3 tables, Manuscript for IEEE Transactions on Intelligent Transportation Systems

Simple Hierarchical Planning with Diffusion

Authors: Chang Chen, Fei Deng, Kenji Kawaguchi, Caglar Gulcehre, Sung** Ahn

Abstract: Diffusion-based generative methods have proven effective in modeling trajectories with offline datasets. However, they often face computational challenges and can falter in generalization, especially in capturing temporal abstractions for long-horizon tasks. To overcome this, we introduce the Hierarchical Diffuser, a simple, fast, yet surprisingly effective planning method combining the advantages… ▽ More Diffusion-based generative methods have proven effective in modeling trajectories with offline datasets. However, they often face computational challenges and can falter in generalization, especially in capturing temporal abstractions for long-horizon tasks. To overcome this, we introduce the Hierarchical Diffuser, a simple, fast, yet surprisingly effective planning method combining the advantages of hierarchical and diffusion-based planning. Our model adopts a "jumpy" planning strategy at the higher level, which allows it to have a larger receptive field but at a lower computational cost -- a crucial factor for diffusion-based planning methods, as we have empirically verified. Additionally, the jumpy sub-goals guide our low-level planner, facilitating a fine-tuning stage and further improving our approach's effectiveness. We conducted empirical evaluations on standard offline reinforcement learning benchmarks, demonstrating our method's superior performance and efficiency in terms of training and planning speed compared to the non-hierarchical Diffuser as well as other hierarchical planning methods. Moreover, we explore our model's generalization capability, particularly on how our method improves generalization capabilities on compositional out-of-distribution tasks. △ Less

Submitted 5 January, 2024; originally announced January 2024.

Understanding Heterogeneity of Automated Vehicles and Its Traffic-level Impact: A Stochastic Behavioral Perspective

Authors: Xinzhi Zhong, Yang Zhou, Soyoung Ahn, Danjue Chen

Abstract: This paper develops a stochastic and unifying framework to examine variability in car-following (CF) dynamics of commercial automated vehicles (AVs) and its direct relation to traffic-level dynamics. The asymmetric behavior (AB) model by Chen at al. (2012a) is extended to accommodate a range of CF behaviors by AVs and compare with the baseline of human-driven vehicles (HDVs). The parameters of the… ▽ More This paper develops a stochastic and unifying framework to examine variability in car-following (CF) dynamics of commercial automated vehicles (AVs) and its direct relation to traffic-level dynamics. The asymmetric behavior (AB) model by Chen at al. (2012a) is extended to accommodate a range of CF behaviors by AVs and compare with the baseline of human-driven vehicles (HDVs). The parameters of the extended AB (EAB) model are calibrated using an adaptive sequential Monte Carlo method for Approximate Bayesian Computation (ABC-ASMC) to stochastically capture various uncertainties including model mismatch resulting from unknown AV CF logic. The estimated posterior distributions of the parameters reveal significant differences in CF behavior (1) between AVs and HDVs, and (2) across AV developers, engine modes, and speed ranges, albeit to a lesser degree. The estimated behavioral patterns and simulation experiments further reveal mixed platoon dynamics in terms of traffic throughout reduction and hysteresis. △ Less

Submitted 30 December, 2023; originally announced January 2024.

Similar but Different: A Survey of Ground Segmentation and Traversability Estimation for Terrestrial Robots

Authors: Hyungtae Lim, Minho Oh, Seungjae Lee, Seunguk Ahn, Hyun Myung

Abstract: With the increasing demand for mobile robots and autonomous vehicles, several approaches for long-term robot navigation have been proposed. Among these techniques, ground segmentation and traversability estimation play important roles in perception and path planning, respectively. Even though these two techniques appear similar, their objectives are different. Ground segmentation divides data into… ▽ More With the increasing demand for mobile robots and autonomous vehicles, several approaches for long-term robot navigation have been proposed. Among these techniques, ground segmentation and traversability estimation play important roles in perception and path planning, respectively. Even though these two techniques appear similar, their objectives are different. Ground segmentation divides data into ground and non-ground elements; thus, it is used as a preprocessing stage to extract objects of interest by rejecting ground points. In contrast, traversability estimation identifies and comprehends areas in which robots can move safely. Nevertheless, some researchers use these terms without clear distinction, leading to misunderstanding the two concepts. Therefore, in this study, we survey related literature and clearly distinguish ground and traversable regions considering four aspects: a) maneuverability of robot platforms, b) position of a robot in the surroundings, c) subset relation of negative obstacles, and d) subset relation of deformable objects. △ Less

Submitted 2 January, 2024; v1 submitted 28 December, 2023; originally announced December 2023.

Comments: 10 pages, 8 figures

Large Language Models in Medical Term Classification and Unexpected Misalignment Between Response and Reasoning

Authors: Xiaodan Zhang, Sandeep Vemulapalli, Nabasmita Talukdar, Sumyeong Ahn, Jiankun Wang, Han Meng, Sardar Mehtab Bin Murtaza, Aakash Ajay Dave, Dmitry Leshchiner, Dimitri F. Joseph, Martin Witteveen-Lane, Dave Chesla, Jiayu Zhou, Bin Chen

Abstract: This study assesses the ability of state-of-the-art large language models (LLMs) including GPT-3.5, GPT-4, Falcon, and LLaMA 2 to identify patients with mild cognitive impairment (MCI) from discharge summaries and examines instances where the models' responses were misaligned with their reasoning. Utilizing the MIMIC-IV v2.2 database, we focused on a cohort aged 65 and older, verifying MCI diagnos… ▽ More This study assesses the ability of state-of-the-art large language models (LLMs) including GPT-3.5, GPT-4, Falcon, and LLaMA 2 to identify patients with mild cognitive impairment (MCI) from discharge summaries and examines instances where the models' responses were misaligned with their reasoning. Utilizing the MIMIC-IV v2.2 database, we focused on a cohort aged 65 and older, verifying MCI diagnoses against ICD codes and expert evaluations. The data was partitioned into training, validation, and testing sets in a 7:2:1 ratio for model fine-tuning and evaluation, with an additional metastatic cancer dataset from MIMIC III used to further assess reasoning consistency. GPT-4 demonstrated superior interpretative capabilities, particularly in response to complex prompts, yet displayed notable response-reasoning inconsistencies. In contrast, open-source models like Falcon and LLaMA 2 achieved high accuracy but lacked explanatory reasoning, underscoring the necessity for further research to optimize both performance and interpretability. The study emphasizes the significance of prompt engineering and the need for further exploration into the unexpected reasoning-response misalignment observed in GPT-4. The results underscore the promise of incorporating LLMs into healthcare diagnostics, contingent upon methodological advancements to ensure accuracy and clinical coherence of AI-generated outputs, thereby improving the trustworthiness of LLMs for medical decision-making. △ Less

Submitted 19 December, 2023; originally announced December 2023.

A Generic Stochastic Hybrid Car-following Model Based on Approximate Bayesian Computation

Authors: Jiwan Jiang, Yang Zhou, Xin Wang, Soyoung Ahn

Abstract: Car following (CF) models are fundamental to describing traffic dynamics. However, the CF behavior of human drivers is highly stochastic and nonlinear. As a result, identifying the best CF model has been challenging and controversial despite decades of research. Introduction of automated vehicles has further complicated this matter as their CF controllers remain proprietary, though their behavior… ▽ More Car following (CF) models are fundamental to describing traffic dynamics. However, the CF behavior of human drivers is highly stochastic and nonlinear. As a result, identifying the best CF model has been challenging and controversial despite decades of research. Introduction of automated vehicles has further complicated this matter as their CF controllers remain proprietary, though their behavior appears different than human drivers. This paper develops a stochastic learning approach to integrate multiple CF models, rather than relying on a single model. The framework is based on approximate Bayesian computation that probabilistically concatenates a pool of CF models based on their relative likelihood of describing observed behavior. The approach, while data-driven, retains physical tractability and interpretability. Evaluation results using two datasets show that the proposed approach can better reproduce vehicle trajectories for both human driven and automated vehicles than any single CF model considered. △ Less

Submitted 26 November, 2023; originally announced December 2023.

Comments: 25 pages, 6 figures

Constraining nucleon effective masses with flow and stop** observables from the S$π$RIT experiment

Authors: C. Y. Tsang, M. Kurata-Nishimura, M. B. Tsang, W. G. Lynch, Y. X. Zhang, J. Barney, J. Estee, G. Jhang, R. Wang, M. Kaneko, J. W. Lee, T. Isobe, T. Murakami, D. S. Ahn, L. Atar, T. Aumann, H. Baba, K. Boretzky, J. Brzychczyk, G. Cerizza, N. Chiga, N. Fukuda, I. Gasparic, B. Hong, A. Horvat , et al. (30 additional authors not shown)

Abstract: Properties of the nuclear equation of state (EoS) can be probed by measuring the dynamical properties of nucleus-nucleus collisions. In this study, we present the directed flow ($v_1$), elliptic flow ($v_2$) and stop** (VarXZ) measured in fixed target Sn + Sn collisions at 270 AMeV with the S$π$RIT Time Projection Chamber. We perform Bayesian analyses in which EoS parameters are varied simultane… ▽ More Properties of the nuclear equation of state (EoS) can be probed by measuring the dynamical properties of nucleus-nucleus collisions. In this study, we present the directed flow ($v_1$), elliptic flow ($v_2$) and stop** (VarXZ) measured in fixed target Sn + Sn collisions at 270 AMeV with the S$π$RIT Time Projection Chamber. We perform Bayesian analyses in which EoS parameters are varied simultaneously within the Improved Quantum Molecular Dynamics-Skyrme (ImQMD-Sky) transport code to obtain a multivariate correlated constraint. The varied parameters include symmetry energy, $S_0$, and slope of the symmetry energy, $L$, at saturation density, isoscalar effective mass, $m_{s}^*/m_{N}$, isovector effective mass, $m_{v}^{*}/m_{N}$ and the in-medium cross-section enhancement factor $η$. We find that the flow and VarXZ observables are sensitive to the splitting of proton and neutron effective masses and the in-medium cross-section. Comparisons of ImQMD-Sky predictions to the S$π$RIT data suggest a narrow range of preferred values for $m_{s}^*/m_{N}$, $m_{v}^{*}/m_{N}$ and $η$. △ Less

Submitted 8 December, 2023; originally announced December 2023.

Triplet Edge Attention for Algorithmic Reasoning

Authors: Yeonjoon Jung, Sungsoo Ahn

Abstract: This work investigates neural algorithmic reasoning to develop neural networks capable of learning from classical algorithms. The main challenge is to develop graph neural networks that are expressive enough to predict the given algorithm outputs while generalizing well to out-of-distribution data. In this work, we introduce a new graph neural network layer called Triplet Edge Attention (TEA), an… ▽ More This work investigates neural algorithmic reasoning to develop neural networks capable of learning from classical algorithms. The main challenge is to develop graph neural networks that are expressive enough to predict the given algorithm outputs while generalizing well to out-of-distribution data. In this work, we introduce a new graph neural network layer called Triplet Edge Attention (TEA), an edge-aware graph attention layer. Our algorithm works by precisely computing edge latent, aggregating multiple triplet messages using edge-based attention. We empirically validate our TEA layer in the CLRS benchmark and demonstrate a $5%$ improvement on average. In particular, we achieve a $30%$ improvement for the string algorithms compared to the state-of-the-art model. △ Less

Submitted 9 December, 2023; originally announced December 2023.

A Simple and Scalable Representation for Graph Generation

Authors: Yunhui Jang, Seul Lee, Sungsoo Ahn

Abstract: Recently, there has been a surge of interest in employing neural networks for graph generation, a fundamental statistical learning problem with critical applications like molecule design and community analysis. However, most approaches encounter significant limitations when generating large-scale graphs. This is due to their requirement to output the full adjacency matrices whose size grows quadra… ▽ More Recently, there has been a surge of interest in employing neural networks for graph generation, a fundamental statistical learning problem with critical applications like molecule design and community analysis. However, most approaches encounter significant limitations when generating large-scale graphs. This is due to their requirement to output the full adjacency matrices whose size grows quadratically with the number of nodes. In response to this challenge, we introduce a new, simple, and scalable graph representation named gap encoded edge list (GEEL) that has a small representation size that aligns with the number of edges. In addition, GEEL significantly reduces the vocabulary size by incorporating the gap encoding and bandwidth restriction schemes. GEEL can be autoregressively generated with the incorporation of node positional encoding, and we further extend GEEL to deal with attributed graphs by designing a new grammar. Our findings reveal that the adoption of this compact representation not only enhances scalability but also bolsters performance by simplifying the graph generation process. We conduct a comprehensive evaluation across ten non-attributed and two molecular graph generation tasks, demonstrating the effectiveness of GEEL. △ Less

Submitted 26 March, 2024; v1 submitted 3 December, 2023; originally announced December 2023.

Comments: International Conference on Learning Representations (ICLR) 2024

Heteroscedastic Uncertainty Estimation for Probabilistic Unsupervised Registration of Noisy Medical Images

Authors: Xiaoran Zhang, Daniel H. Pak, Shawn S. Ahn, Xiaoxiao Li, Chenyu You, Lawrence Staib, Albert J. Sinusas, Alex Wong, James S. Duncan

Abstract: This paper proposes a heteroscedastic uncertainty estimation framework for unsupervised medical image registration. Existing methods rely on objectives (e.g. mean-squared error) that assume a uniform noise level across the image, disregarding the heteroscedastic and input-dependent characteristics of noise distribution in real-world medical images. This further introduces noisy gradients due to un… ▽ More This paper proposes a heteroscedastic uncertainty estimation framework for unsupervised medical image registration. Existing methods rely on objectives (e.g. mean-squared error) that assume a uniform noise level across the image, disregarding the heteroscedastic and input-dependent characteristics of noise distribution in real-world medical images. This further introduces noisy gradients due to undesired penalization on outliers, causing unnatural deformation and performance degradation. To mitigate this, we propose an adaptive weighting scheme with a relative $γ$-exponentiated signal-to-noise ratio (SNR) for the displacement estimator after modeling the heteroscedastic noise using a separate variance estimator to prevent the model from being driven away by spurious gradients from error residuals, leading to more accurate displacement estimation. To illustrate the versatility and effectiveness of the proposed method, we tested our framework on two representative registration architectures across three medical image datasets. Our proposed framework consistently outperforms other baselines both quantitatively and qualitatively while also providing accurate and sensible uncertainty measures. Paired t-tests show that our improvements in registration accuracy are statistically significant. The code will be publicly available at \url{https://voldemort108x.github.io/hetero_uncertainty/}. △ Less

Submitted 30 November, 2023; originally announced December 2023.

A superconducting tensor detector for mid-frequency gravitational waves: its multi-channel nature and main astrophysical targets

Authors: Yeong-Bok Bae, Chan Park, Edwin J. Son, Sang-Hyeon Ahn, Minjoong Jeong, Gungwon Kang, Chunglee Kim, Dong Lak Kim, Jaewan Kim, Whansun Kim, Hyung Mok Lee, Yong-Ho Lee, Ronald S. Norton, John J. Oh, Sang Hoon Oh, Ho Jung Paik

Abstract: Mid-frequency band gravitational-wave detectors will be complementary for the existing Earth-based detectors (sensitive above 10 Hz or so) and the future space-based detectors such as LISA, which will be sensitive below around 10 mHz. A ground-based superconducting omnidirectional gravitational radiation observatory (SOGRO) has recently been proposed along with several design variations for the fr… ▽ More Mid-frequency band gravitational-wave detectors will be complementary for the existing Earth-based detectors (sensitive above 10 Hz or so) and the future space-based detectors such as LISA, which will be sensitive below around 10 mHz. A ground-based superconducting omnidirectional gravitational radiation observatory (SOGRO) has recently been proposed along with several design variations for the frequency band of 0.1 to 10 Hz. For three conceptual designs of SOGRO (e.g., pSOGRO, SOGRO and aSOGRO), we examine their multi-channel natures, sensitivities and science cases. One of the key characteristics of the SOGRO concept is its six detection channels. The response functions of each channel are calculated for all possible gravitational wave polarizations including scalar and vector modes. Combining these response functions, we also confirm the omnidirectional nature of SOGRO. Hence, even a single SOGRO detector will be able to determine the position of a source and polarizations of gravitational waves, if detected. Taking into account SOGRO's sensitivity and technical requirements, two main targets are most plausible: gravitational waves from compact binaries and stochastic backgrounds. Based on assumptions we consider in this work, detection rates for intermediate-mass binary black holes (in the mass range of hundreds up to $10^{4}$ $M_\odot$) are expected to be $0.0014-2.5 \,\, {\rm yr}^{-1}$. In order to detect stochastic gravitational wave background, multiple detectors are required. Two aSOGRO detector networks may be able to put limits on the stochastic background beyond the indirect limit from cosmological observations. △ Less

Submitted 29 November, 2023; originally announced November 2023.

Comments: 35 pages, 8 figures, 4 tables

Active Prompt Learning in Vision Language Models

Authors: Jihwan Bang, Sumyeong Ahn, Jae-Gil Lee

Abstract: Pre-trained Vision Language Models (VLMs) have demonstrated notable progress in various zero-shot tasks, such as classification and retrieval. Despite their performance, because improving performance on new tasks requires task-specific knowledge, their adaptation is essential. While labels are needed for the adaptation, acquiring them is typically expensive. To overcome this challenge, active lear… ▽ More Pre-trained Vision Language Models (VLMs) have demonstrated notable progress in various zero-shot tasks, such as classification and retrieval. Despite their performance, because improving performance on new tasks requires task-specific knowledge, their adaptation is essential. While labels are needed for the adaptation, acquiring them is typically expensive. To overcome this challenge, active learning, a method of achieving a high performance by obtaining labels for a small number of samples from experts, has been studied. Active learning primarily focuses on selecting unlabeled samples for labeling and leveraging them to train models. In this study, we pose the question, "how can the pre-trained VLMs be adapted under the active learning framework?" In response to this inquiry, we observe that (1) simply applying a conventional active learning framework to pre-trained VLMs even may degrade performance compared to random selection because of the class imbalance in labeling candidates, and (2) the knowledge of VLMs can provide hints for achieving the balance before labeling. Based on these observations, we devise a novel active learning framework for VLMs, denoted as PCB. To assess the effectiveness of our approach, we conduct experiments on seven different real-world datasets, and the results demonstrate that PCB surpasses conventional active learning and random sampling methods. Code will be available in https://github.com/kaist-dmlab/pcb . △ Less

Submitted 21 March, 2024; v1 submitted 18 November, 2023; originally announced November 2023.

Comments: accepted at CVPR 2024

Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World Models

Authors: Yeongbin Kim, Gautam Singh, Junyeong Park, Caglar Gulcehre, Sung** Ahn

Abstract: Systematic compositionality, or the ability to adapt to novel situations by creating a mental model of the world using reusable pieces of knowledge, remains a significant challenge in machine learning. While there has been considerable progress in the language domain, efforts towards systematic visual imagination, or envisioning the dynamical implications of a visual observation, are in their infa… ▽ More Systematic compositionality, or the ability to adapt to novel situations by creating a mental model of the world using reusable pieces of knowledge, remains a significant challenge in machine learning. While there has been considerable progress in the language domain, efforts towards systematic visual imagination, or envisioning the dynamical implications of a visual observation, are in their infancy. We introduce the Systematic Visual Imagination Benchmark (SVIB), the first benchmark designed to address this problem head-on. SVIB offers a novel framework for a minimal world modeling problem, where models are evaluated based on their ability to generate one-step image-to-image transformations under a latent world dynamics. The framework provides benefits such as the possibility to jointly optimize for systematic perception and imagination, a range of difficulty levels, and the ability to control the fraction of possible factor combinations used during training. We provide a comprehensive evaluation of various baseline models on SVIB, offering insight into the current state-of-the-art in systematic visual imagination. We hope that this benchmark will help advance visual systematic compositionality. △ Less

Submitted 15 November, 2023; originally announced November 2023.

Comments: Published as a conference paper at NeurIPS 2023. The first two authors contributed equally. To download the benchmark, visit https://systematic-visual-imagination.github.io

Fine tuning Pre trained Models for Robustness Under Noisy Labels

Authors: Sumyeong Ahn, Sihyeon Kim, Jongwoo Ko, Se-Young Yun

Abstract: The presence of noisy labels in a training dataset can significantly impact the performance of machine learning models. To tackle this issue, researchers have explored methods for Learning with Noisy Labels to identify clean samples and reduce the influence of noisy labels. However, constraining the influence of a certain portion of the training dataset can result in a reduction in overall general… ▽ More The presence of noisy labels in a training dataset can significantly impact the performance of machine learning models. To tackle this issue, researchers have explored methods for Learning with Noisy Labels to identify clean samples and reduce the influence of noisy labels. However, constraining the influence of a certain portion of the training dataset can result in a reduction in overall generalization performance. To alleviate this, recent studies have considered the careful utilization of noisy labels by leveraging huge computational resources. Therefore, the increasing training cost necessitates a reevaluation of efficiency. In other areas of research, there has been a focus on develo** fine-tuning techniques for large pre-trained models that aim to achieve both high generalization performance and efficiency. However, these methods have mainly concentrated on clean datasets, and there has been limited exploration of the noisy label scenario. In this research, our aim is to find an appropriate way to fine-tune pre-trained models for noisy labeled datasets. To achieve this goal, we investigate the characteristics of pre-trained models when they encounter noisy datasets. Through empirical analysis, we introduce a novel algorithm called TURN, which robustly and efficiently transfers the prior knowledge of pre-trained models. The algorithm consists of two main steps: (1) independently tuning the linear classifier to protect the feature extractor from being distorted by noisy labels, and (2) reducing the noisy label ratio and fine-tuning the entire model based on the noise-reduced dataset to adapt it to the target dataset. The proposed algorithm has been extensively tested and demonstrates efficient yet improved denoising performance on various benchmarks compared to previous methods. △ Less

Submitted 24 October, 2023; originally announced October 2023.

Comments: 10 pages (17 pages including supplementary)

MSC Class: Computer Science; Artificial Intelligence

Exploring the Impact of Corpus Diversity on Financial Pretrained Language Models

Authors: Jaeyoung Choe, Keonwoong Noh, Nayeon Kim, Seyun Ahn, Woohwan Jung

Abstract: Over the past few years, various domain-specific pretrained language models (PLMs) have been proposed and have outperformed general-domain PLMs in specialized areas such as biomedical, scientific, and clinical domains. In addition, financial PLMs have been studied because of the high economic impact of financial data analysis. However, we found that financial PLMs were not pretrained on sufficient… ▽ More Over the past few years, various domain-specific pretrained language models (PLMs) have been proposed and have outperformed general-domain PLMs in specialized areas such as biomedical, scientific, and clinical domains. In addition, financial PLMs have been studied because of the high economic impact of financial data analysis. However, we found that financial PLMs were not pretrained on sufficiently diverse financial data. This lack of diverse training data leads to a subpar generalization performance, resulting in general-purpose PLMs, including BERT, often outperforming financial PLMs on many downstream tasks. To address this issue, we collected a broad range of financial corpus and trained the Financial Language Model (FiLM) on these diverse datasets. Our experimental results confirm that FiLM outperforms not only existing financial PLMs but also general domain PLMs. Furthermore, we provide empirical evidence that this improvement can be achieved even for unseen corpus groups. △ Less

Submitted 20 October, 2023; originally announced October 2023.

Comments: Accepted to EMNLP 2023 (Findings)