Showing 1–50 of 488 results for author: Baek, S

CLOi-Mapper: Consistent, Lightweight, Robust, and Incremental Mapper With Embedded Systems for Commercial Robot Services

Authors: DongKi Noh, Hyungtae Lim, Gyuho Eoh, Duckyu Choi, Jeongsik Choi, Hyunjun Lim, SeungMin Baek, Hyun Myung

Abstract: In commercial autonomous service robots with several form factors, simultaneous localization and map** (SLAM) is an essential technology for providing proper services such as cleaning and guidance. Such robots require SLAM algorithms suitable for specific applications and environments. Hence, several SLAM frameworks have been proposed to address various requirements in the past decade. However,… ▽ More In commercial autonomous service robots with several form factors, simultaneous localization and map** (SLAM) is an essential technology for providing proper services such as cleaning and guidance. Such robots require SLAM algorithms suitable for specific applications and environments. Hence, several SLAM frameworks have been proposed to address various requirements in the past decade. However, we have encountered challenges in implementing recent innovative frameworks when handling service robots with low-end processors and insufficient sensor data, such as low-resolution 2D LiDAR sensors. Specifically, regarding commercial robots, consistent performance in different hardware configurations and environments is more crucial than the performance dedicated to specific sensors or environments. Therefore, we propose a) a multi-stage %hierarchical approach for global pose estimation in embedded systems; b) a graph generation method with zero constraints for synchronized sensors; and c) a robust and memory-efficient method for long-term pose-graph optimization. As verified in in-home and large-scale indoor environments, the proposed method yields consistent global pose estimation for services in commercial fields. Furthermore, the proposed method exhibits potential commercial viability considering the consistent performance verified via mass production and long-term (> 5 years) operation. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Journal ref: IEEE Robotics and Automation Letters, 2024

Superconducting phase diagram in Bi$_x$Ni$_{1-x}$ thin films$\colon$ the effects of Bi stoichiometry on superconductivity

Authors: Jihun Park, Jarryd A. Horn, Dylan J. Kirsch, Rohit K. Pant, Hyeok Yoon, Sungha Baek, Suchismita Sarker, Apurva Mehta, Xiaohang Zhang, Seunghun Lee, Richard Greene, Johnpierre Paglione, Ichiro Takeuchi

Abstract: The Bi${-}$Ni binary system has been of interest due to possible unconventional superconductivity aroused therein, such as time-reversal symmetry breaking in Bi/Ni bilayers or the coexistence of superconductivity and ferromagnetism in Bi$_3$Ni crystals. While Ni acts as a ferromagnetic element in such systems, the role of strong spin-orbit-coupling element Bi in superconductivity has remained unex… ▽ More The Bi${-}$Ni binary system has been of interest due to possible unconventional superconductivity aroused therein, such as time-reversal symmetry breaking in Bi/Ni bilayers or the coexistence of superconductivity and ferromagnetism in Bi$_3$Ni crystals. While Ni acts as a ferromagnetic element in such systems, the role of strong spin-orbit-coupling element Bi in superconductivity has remained unexplored. In this work, we systematically studied the effects of Bi stoichiometry on the superconductivity of Bi$_x$Ni$_{1-x}$ thin films (${x} \approx$ 0.5 to 0.9) fabricated via a composition-spread approach. The superconducting phase map of Bi$_x$Ni$_{1-x}$ thin films exhibited a superconducting composition region attributable to the intermetallic Bi$_3$Ni phase with different amount of excess Bi, revealed by synchrotron X-ray diffraction analysis. Interestingly, the mixed phase region with Bi$_3$Ni and Bi showed unusual increases in the superconducting transition temperature and residual resistance ratio as more Bi impurities were included, with the maximum ${T}_{c}$ ($=$ 4.2 K) observed at $x \approx$ 0.79. A correlation analysis of structural, electrical, and magneto-transport characteristics across the composition variation revealed that the unusual superconducting $"$dome$"$ is due to two competing roles of Bi$\colon$ impurity scattering and carrier do**. We found that the carrier do** effect is dominant in the mild do** regime (0.74 $\leq {x} \leq$ 0.79), while impurity scattering becomes more pronounced at larger Bi stoichiometry. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Augmenting Query and Passage for Retrieval-Augmented Generation using LLMs for Open-Domain Question Answering

Authors: Minsang Kim, Cheoneum Park, Seungjun Baek

Abstract: Retrieval-augmented generation (RAG) has received much attention for Open-domain question-answering (ODQA) tasks as a means to compensate for the parametric knowledge of large language models (LLMs). While previous approaches focused on processing retrieved passages to remove irrelevant context, they still rely heavily on the quality of retrieved passages which can degrade if the question is ambig… ▽ More Retrieval-augmented generation (RAG) has received much attention for Open-domain question-answering (ODQA) tasks as a means to compensate for the parametric knowledge of large language models (LLMs). While previous approaches focused on processing retrieved passages to remove irrelevant context, they still rely heavily on the quality of retrieved passages which can degrade if the question is ambiguous or complex. In this paper, we propose a simple yet efficient method called question and passage augmentation via LLMs for open-domain QA. Our method first decomposes the original questions into multiple-step sub-questions. By augmenting the original question with detailed sub-questions and planning, we are able to make the query more specific on what needs to be retrieved, improving the retrieval performance. In addition, to compensate for the case where the retrieved passages contain distracting information or divided opinions, we augment the retrieved passages with self-generated passages by LLMs to guide the answer extraction. Experimental results show that the proposed scheme outperforms the previous state-of-the-art and achieves significant performance gain over existing RAG methods. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Measuring Sample Importance in Data Pruning for Training LLMs from a Data Compression Perspective

Authors: Minsang Kim, Seungjun Baek

Abstract: Compute-efficient training of large language models (LLMs) has become an important research problem. In this work, we consider data pruning as a method of data-efficient training of LLMs, where we take a data compression view on data pruning. We argue that the amount of information of a sample, or the achievable compression on its description length, represents its sample importance. The key idea… ▽ More Compute-efficient training of large language models (LLMs) has become an important research problem. In this work, we consider data pruning as a method of data-efficient training of LLMs, where we take a data compression view on data pruning. We argue that the amount of information of a sample, or the achievable compression on its description length, represents its sample importance. The key idea is that, less informative samples are likely to contain redundant information, and thus should be pruned first. We leverage log-likelihood function of trained models as a surrogate to measure information content of samples. Experiments reveal a surprising insight that information-based pruning can enhance the generalization capability of the model, improves upon language modeling and downstream tasks as compared to the model trained on the entire dataset. △ Less

Submitted 20 June, 2024; v1 submitted 20 June, 2024; originally announced June 2024.

Reachability Analysis for Linear Systems with Uncertain Parameters using Polynomial Zonotopes

Authors: Yushen Huang, Ertai Luo, Stanley Bak, Yifan Sun

Abstract: In real world applications, uncertain parameters are the rule rather than the exception. We present a reachability algorithm for linear systems with uncertain parameters and inputs using set propagation of polynomial zonotopes. In contrast to previous methods, our approach is able to tightly capture the non-convexity of the reachable set. Building up on our main result, we show how our reachabilit… ▽ More In real world applications, uncertain parameters are the rule rather than the exception. We present a reachability algorithm for linear systems with uncertain parameters and inputs using set propagation of polynomial zonotopes. In contrast to previous methods, our approach is able to tightly capture the non-convexity of the reachable set. Building up on our main result, we show how our reachability algorithm can be extended to handle linear time-varying systems as well as linear systems with time-varying parameters. Moreover, our approach opens up new possibilities for reachability analysis of linear time-invariant systems, nonlinear systems, and hybrid systems. We compare our approach to other state of the art methods, with superior tightness on two benchmarks including a 9-dimensional vehicle platooning system. Moreover, as part of the journal extension, we investigate through a polynomial zonotope with special structure named multi-affine zonotopes and its optimization problem. We provide the corresponding optimization algorithm and experiment over the examples obatined from two benchmark systems, showing the efficiency and scalability comparing to the state of the art method for handling such type of set representation. △ Less

Submitted 16 June, 2024; originally announced June 2024.

Polarization Wavefront Lidar: Learning Large Scene Reconstruction from Polarized Wavefronts

Authors: Dominik Scheuble, Chenyang Lei, Seung-Hwan Baek, Mario Bijelic, Felix Heide

Abstract: Lidar has become a cornerstone sensing modality for 3D vision, especially for large outdoor scenarios and autonomous driving. Conventional lidar sensors are capable of providing centimeter-accurate distance information by emitting laser pulses into a scene and measuring the time-of-flight (ToF) of the reflection. However, the polarization of the received light that depends on the surface orientati… ▽ More Lidar has become a cornerstone sensing modality for 3D vision, especially for large outdoor scenarios and autonomous driving. Conventional lidar sensors are capable of providing centimeter-accurate distance information by emitting laser pulses into a scene and measuring the time-of-flight (ToF) of the reflection. However, the polarization of the received light that depends on the surface orientation and material properties is usually not considered. As such, the polarization modality has the potential to improve scene reconstruction beyond distance measurements. In this work, we introduce a novel long-range polarization wavefront lidar sensor (PolLidar) that modulates the polarization of the emitted and received light. Departing from conventional lidar sensors, PolLidar allows access to the raw time-resolved polarimetric wavefronts. We leverage polarimetric wavefronts to estimate normals, distance, and material properties in outdoor scenarios with a novel learned reconstruction method. To train and evaluate the method, we introduce a simulated and real-world long-range dataset with paired raw lidar data, ground truth distance, and normal maps. We find that the proposed method improves normal and distance reconstruction by 53\% mean angular error and 41\% mean absolute error compared to existing shape-from-polarization (SfP) and ToF methods. Code and data are open-sourced at https://light.princeton.edu/pollidar. △ Less

Submitted 11 June, 2024; v1 submitted 5 June, 2024; originally announced June 2024.

Comments: Accepted at CVPR 2024; Project Website: https://light.princeton.edu/publication/pollidar

Verification of Neural Network Control Systems in Continuous Time

Authors: Ali ArjomandBigdeli, Andrew Mata, Stanley Bak

Abstract: Neural network controllers are currently being proposed for use in many safety-critical tasks. Most analysis methods for neural network control systems assume a fixed control period. In control theory, higher frequency usually improves performance. However, for current analysis methods, increasing the frequency complicates verification. In the limit, when actuation is performed continuously, no ex… ▽ More Neural network controllers are currently being proposed for use in many safety-critical tasks. Most analysis methods for neural network control systems assume a fixed control period. In control theory, higher frequency usually improves performance. However, for current analysis methods, increasing the frequency complicates verification. In the limit, when actuation is performed continuously, no existing neural network control systems verification methods are able to analyze the system. In this work, we develop the first verification method for continuously-actuated neural network control systems. We accomplish this by adding a level of abstraction to model the neural network controller. The abstraction is a piecewise linear model with added noise to account for local linearization error. The soundness of the abstraction can be checked using open-loop neural network verification tools, although we demonstrate bottlenecks in existing tools when handling the required specifications. We demonstrate the approach's efficacy by applying it to a vision-based autonomous airplane taxiing system and compare with a fixed frequency analysis baseline. △ Less

Submitted 31 May, 2024; originally announced June 2024.

Comments: 17 pages, 7 figures, Proceedings of the 7th International Symposium on AI Verification (SAIV)

Scalable Surrogate Verification of Image-based Neural Network Control Systems using Composition and Unrolling

Authors: Feiyang Cai, Chuchu Fan, Stanley Bak

Abstract: Verifying safety of neural network control systems that use images as input is a difficult problem because, from a given system state, there is no known way to mathematically model what images are possible in the real-world. We build on recent work that considers a surrogate verification approach, training a conditional generative adversarial network (cGAN) as an image generator in place of the re… ▽ More Verifying safety of neural network control systems that use images as input is a difficult problem because, from a given system state, there is no known way to mathematically model what images are possible in the real-world. We build on recent work that considers a surrogate verification approach, training a conditional generative adversarial network (cGAN) as an image generator in place of the real world. This enables set-based formal analysis of the closed-loop system, providing analysis beyond simulation and testing. While existing work is effective on small examples, excessive overapproximation both within a single control period and across multiple control periods limits its scalability. We propose approaches to overcome these two sources of error. First, we overcome one-step error by composing the system's dynamics along with the cGAN and neural network controller, without losing the dependencies between input states and the control outputs as in the monotonic analysis of the system dynamics. Second, we reduce multi-step error by repeating the single-step composition, essentially unrolling multiple steps of the control loop into a large neural network. We then leverage existing network verification tools to compute accurate reachable sets for multiple steps, avoiding the accumulation of abstraction error at each step. We demonstrate the effectiveness of our approach in terms of both accuracy and scalability using two case studies: an autonomous aircraft taxiing system and an advanced emergency braking system. On the aircraft taxiing system, the converged reachable set is 175% larger using the prior baseline method compared with our proposed approach. On the emergency braking system, with 24x the number of image output variables from the cGAN, the baseline method fails to prove any states are safe, whereas our improvements enable set-based safety analysis. △ Less

Submitted 28 May, 2024; originally announced May 2024.

The Realization of a Gas Puff Imaging System on the Wendelstein 7-X Stellarator

Authors: J. L. Terry, A. von Stechow, S. G. Baek, S. B. Ballinger, O. Grulke, C. von Sehren, R. Laube, C. Killer, F. Scharmer, K. J. Brunner, J. Knauer, S. Bois, the W7-X Team

Abstract: A system for studying the spatio-temporal dynamics of fluctuations in the boundary of the W7-X plasma using the Gas-Puff Imaging (GPI) technique has been designed, constructed, installed, and operated. This GPI system addresses a number of challenges specific to long-pulse superconducting devices like W7-X, including the long distance between the plasma and the vacuum vessel wall, the long distanc… ▽ More A system for studying the spatio-temporal dynamics of fluctuations in the boundary of the W7-X plasma using the Gas-Puff Imaging (GPI) technique has been designed, constructed, installed, and operated. This GPI system addresses a number of challenges specific to long-pulse superconducting devices like W7-X, including the long distance between the plasma and the vacuum vessel wall, the long distance between the plasma and diagnostic ports, the range of last closed flux surface locations for different magnetic configurations in W7-X, and management of heat loads on the system's plasma-facing components. The system features a pair of "converging-diverging" nozzles for partially collimating the gas puffed locally $\approx$135 mm radially outboard of the plasma boundary, a pop-up turning mirror for viewing the gas puff emission from the side (also acting as a shutter for the re-entrant vacuum window), and a high-throughput optical system that collects visible emission resulting from the interaction between the puffed gas and the plasma and directs it along a water-cooled re-entrant tube directly onto the 8 x 16 pixel detector array of the fast camera. The DEGAS 2 neutrals code was used to simulate the H$_α$ (656 nm) and the HeI (587 nm) line emission expected from well-characterized gas-puffs of H$_2$ and He and excited within typical edge plasma profiles in W7-X, thereby predicting line brightnesses used to reduce the risks associated with system sensitivity and placement of the field of view. Operation of GPI on W7-X shows excellent signal to noise ratios (>100) over the field of view for minimally perturbing gas puffs. The GPI system provides detailed measurements of the 2-dimensional (radial and poloidal) dynamics of plasma fluctuations in the W7-X edge, scrape-off layer, and in and around the magnetic islands that make up the island divertor configuration employed on W7-X. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Comments: 30 pages, 23 figures, submitted to Review of Scientific Instruments

3D bulk field theories for 2D non-unitary N=1 supersymmetric minimal models

Authors: Seungjoo Baek, Dongmin Gang

Abstract: We propose bulk 3D N=4 rank-0 superconformal field theories, which are related to 2D N=1 supersymmetric minimal models, SM(2, ...) and SM(3, ...), via recently discovered non-unitary bulk-boundary correspondence. The correspondence relates a 3D N=4 rank-0 superconformal field theory to 2D chiral rational conformal field theories. A topologically twisted theory of the rank-0 SCFT supports the ratio… ▽ More We propose bulk 3D N=4 rank-0 superconformal field theories, which are related to 2D N=1 supersymmetric minimal models, SM(2, ...) and SM(3, ...), via recently discovered non-unitary bulk-boundary correspondence. The correspondence relates a 3D N=4 rank-0 superconformal field theory to 2D chiral rational conformal field theories. A topologically twisted theory of the rank-0 SCFT supports the rational chiral algebra at the boundary upon a proper choice of boundary condition. We test the proposal by checking several non-trivial dictionaries of the correspondence. △ Less

Submitted 3 June, 2024; v1 submitted 9 May, 2024; originally announced May 2024.

Comments: 32 pages, 3 figures

DRAMScope: Uncovering DRAM Microarchitecture and Characteristics by Issuing Memory Commands

Authors: Hwayong Nam, Seungmin Baek, Minbok Wi, Michael Jaemin Kim, Jaehyun Park, Chihun Song, Nam Sung Kim, Jung Ho Ahn

Abstract: The demand for precise information on DRAM microarchitectures and error characteristics has surged, driven by the need to explore processing in memory, enhance reliability, and mitigate security vulnerability. Nonetheless, DRAM manufacturers have disclosed only a limited amount of information, making it difficult to find specific information on their DRAM microarchitectures. This paper addresses t… ▽ More The demand for precise information on DRAM microarchitectures and error characteristics has surged, driven by the need to explore processing in memory, enhance reliability, and mitigate security vulnerability. Nonetheless, DRAM manufacturers have disclosed only a limited amount of information, making it difficult to find specific information on their DRAM microarchitectures. This paper addresses this gap by presenting more rigorous findings on the microarchitectures of commodity DRAM chips and their impacts on the characteristics of activate-induced bitflips (AIBs), such as RowHammer and RowPress. The previous studies have also attempted to understand the DRAM microarchitectures and associated behaviors, but we have found some of their results to be misled by inaccurate address map** and internal data swizzling, or lack of a deeper understanding of the modern DRAM cell structure. For accurate and efficient reverse-engineering, we use three tools: AIBs, retention time test, and RowCopy, which can be cross-validated. With these three tools, we first take a macroscopic view of modern DRAM chips to uncover the size, structure, and operation of their subarrays, memory array tiles (MATs), and rows. Then, we analyze AIB characteristics based on the microscopic view of the DRAM microarchitecture, such as 6F^2 cell layout, through which we rectify misunderstandings regarding AIBs and discover a new data pattern that accelerates AIBs. Lastly, based on our findings at both macroscopic and microscopic levels, we identify previously unknown AIB vulnerabilities and propose a simple yet effective protection solution. △ Less

Submitted 3 May, 2024; originally announced May 2024.

Comments: To appear at the 51st IEEE/ACM International Symposium on Computer Architecture (ISCA)

Exact Cluster Dynamics of Indirect Reciprocity in Complete Graphs

Authors: Minwoo Bae, Takashi Shimada, Seung Ki Baek

Abstract: Heider's balance theory emphasizes cognitive consistency in assessing others, as is expressed by ``The enemy of my enemy is my friend.'' At the same time, the theory of indirect reciprocity provides us with a dynamical framework to study how to assess others based on their actions as well as how to act toward them based on the assessments. Well-known are the `leading eight' from L1 to L8, the eigh… ▽ More Heider's balance theory emphasizes cognitive consistency in assessing others, as is expressed by ``The enemy of my enemy is my friend.'' At the same time, the theory of indirect reciprocity provides us with a dynamical framework to study how to assess others based on their actions as well as how to act toward them based on the assessments. Well-known are the `leading eight' from L1 to L8, the eight norms for assessment and action to foster cooperation in social dilemmas while resisting the invasion of mutant norms prescribing alternative actions. In this work, we begin by showing that balance is equivalent to stationarity of dynamics only for L4 and L6 (Stern Judging) among the leading eight. Stern Judging reflects an intuitive idea that good merits reward whereas evil warrants punishment. By analyzing the dynamics of Stern Judging in complete graphs, we prove that this norm almost always segregates the graph into two mutually hostile groups as the graph size grows. We then compare L4 with Stern Judging: The only difference of L4 is that a good player's cooperative action toward a bad one is regarded as good. This subtle difference transforms large populations governed by L4 to a ``paradise'' where cooperation prevails and positive assessments abound. Our study thus helps us understand the relationship between individual norms and their emergent consequences at a population level, shedding light on the nuanced interplay between cognitive consistency and segregation dynamics. △ Less

Submitted 7 May, 2024; v1 submitted 24 April, 2024; originally announced April 2024.

Generalizable Novel-View Synthesis using a Stereo Camera

Authors: Haechan Lee, Wonjoon **, Seung-Hwan Baek, Sunghyun Cho

Abstract: In this paper, we propose the first generalizable view synthesis approach that specifically targets multi-view stereo-camera images. Since recent stereo matching has demonstrated accurate geometry prediction, we introduce stereo matching into novel-view synthesis for high-quality geometry reconstruction. To this end, this paper proposes a novel framework, dubbed StereoNeRF, which integrates stereo… ▽ More In this paper, we propose the first generalizable view synthesis approach that specifically targets multi-view stereo-camera images. Since recent stereo matching has demonstrated accurate geometry prediction, we introduce stereo matching into novel-view synthesis for high-quality geometry reconstruction. To this end, this paper proposes a novel framework, dubbed StereoNeRF, which integrates stereo matching into a NeRF-based generalizable view synthesis approach. StereoNeRF is equipped with three key components to effectively exploit stereo matching in novel-view synthesis: a stereo feature extractor, a depth-guided plane-swee**, and a stereo depth loss. Moreover, we propose the StereoNVS dataset, the first multi-view dataset of stereo-camera images, encompassing a wide variety of both real and synthetic scenes. Our experimental results demonstrate that StereoNeRF surpasses previous approaches in generalizable view synthesis. △ Less

Submitted 21 April, 2024; originally announced April 2024.

Comments: Accepted to CVPR 2024. Project page URL: https://**wonjoon.github.io/stereonerf/

HyperCLOVA X Technical Report

Authors: Kang Min Yoo, Jaegeun Han, Sookyo In, Heewon Jeon, Jisu Jeong, Jaewook Kang, Hyunwook Kim, Kyung-Min Kim, Munhyong Kim, Sungju Kim, Donghyun Kwak, Hanock Kwak, Se Jung Kwon, Bado Lee, Dongsoo Lee, Gichang Lee, Jooho Lee, Baeseong Park, Seong** Shin, Joonsang Yu, Seolki Baek, Sumin Byeon, Eungsup Cho, Dooseok Choe, Jeesung Han , et al. (371 additional authors not shown)

Abstract: We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment t… ▽ More We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment to responsible AI. The model is evaluated across various benchmarks, including comprehensive reasoning, knowledge, commonsense, factuality, coding, math, chatting, instruction-following, and harmlessness, in both Korean and English. HyperCLOVA X exhibits strong reasoning capabilities in Korean backed by a deep understanding of the language and cultural nuances. Further analysis of the inherent bilingual nature and its extension to multilingualism highlights the model's cross-lingual proficiency and strong generalization ability to untargeted languages, including machine translation between several language pairs and cross-lingual inference tasks. We believe that HyperCLOVA X can provide helpful guidance for regions or countries in develo** their sovereign LLMs. △ Less

Submitted 13 April, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Comments: 44 pages; updated authors list and fixed author names

Gyro-based Neural Single Image Deblurring

Authors: Heemin Yang, Jaesung Rim, Seungyong Lee, Seung-Hwan Baek, Sunghyun Cho

Abstract: In this paper, we present GyroDeblurNet, a novel single image deblurring method that utilizes a gyro sensor to effectively resolve the ill-posedness of image deblurring. The gyro sensor provides valuable information about camera motion during exposure time that can significantly improve deblurring quality. However, effectively exploiting real-world gyro data is challenging due to significant error… ▽ More In this paper, we present GyroDeblurNet, a novel single image deblurring method that utilizes a gyro sensor to effectively resolve the ill-posedness of image deblurring. The gyro sensor provides valuable information about camera motion during exposure time that can significantly improve deblurring quality. However, effectively exploiting real-world gyro data is challenging due to significant errors from various sources including sensor noise, the disparity between the positions of a camera module and a gyro sensor, the absence of translational motion information, and moving objects whose motions cannot be captured by a gyro sensor. To handle gyro error, GyroDeblurNet is equipped with two novel neural network blocks: a gyro refinement block and a gyro deblurring block. The gyro refinement block refines the error-ridden gyro data using the blur information from the input image. On the other hand, the gyro deblurring block removes blur from the input image using the refined gyro data and further compensates for gyro error by leveraging the blur information from the input image. For training a neural network with erroneous gyro data, we propose a training strategy based on the curriculum learning. We also introduce a novel gyro data embedding scheme to represent real-world intricate camera shakes. Finally, we present a synthetic dataset and a real dataset for the training and evaluation of gyro-based single image deblurring. Our experiments demonstrate that our approach achieves state-of-the-art deblurring quality by effectively utilizing erroneous gyro data. △ Less

Submitted 8 April, 2024; v1 submitted 1 April, 2024; originally announced April 2024.

Comments: 14 pages, 11 figures

Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction

Authors: Junuk Cha, Jihyeon Kim, Jae Shin Yoon, Seungryul Baek

Abstract: This paper introduces the first text-guided work for generating the sequence of hand-object interaction in 3D. The main challenge arises from the lack of labeled data where existing ground-truth datasets are nowhere near generalizable in interaction type and object category, which inhibits the modeling of diverse 3D hand-object interaction with the correct physical implication (e.g., contacts and… ▽ More This paper introduces the first text-guided work for generating the sequence of hand-object interaction in 3D. The main challenge arises from the lack of labeled data where existing ground-truth datasets are nowhere near generalizable in interaction type and object category, which inhibits the modeling of diverse 3D hand-object interaction with the correct physical implication (e.g., contacts and semantics) from text prompts. To address this challenge, we propose to decompose the interaction generation task into two subtasks: hand-object contact generation; and hand-object motion generation. For contact generation, a VAE-based network takes as input a text and an object mesh, and generates the probability of contacts between the surfaces of hands and the object during the interaction. The network learns a variety of local geometry structure of diverse objects that is independent of the objects' category, and thus, it is applicable to general objects. For motion generation, a Transformer-based diffusion model utilizes this 3D contact map as a strong prior for generating physically plausible hand-object motion as a function of text prompts by learning from the augmented labeled dataset; where we annotate text labels from many existing 3D hand and object motion data. Finally, we further introduce a hand refiner module that minimizes the distance between the object surface and hand joints to improve the temporal stability of the object-hand contacts and to suppress the penetration artifacts. In the experiments, we demonstrate that our method can generate more realistic and diverse interactions compared to other baseline methods. We also show that our method is applicable to unseen objects. We will release our model and newly labeled data as a strong foundation for future research. Codes and data are available in: https://github.com/JunukCha/Text2HOI. △ Less

Submitted 1 April, 2024; v1 submitted 31 March, 2024; originally announced April 2024.

Comments: Accepted to CVPR 2024

Rindler Fluids from Gravitational Shockwaves

Authors: Sang-Eon Bak, Cynthia Keeler, Yiwen Zhang, Kathryn M. Zurek

Abstract: We study a correspondence between gravitational shockwave geometry and its fluid description near a Rindler horizon in Minkowski spacetime. Utilizing the Petrov classification that describes algebraic symmetries for Lorentzian spaces, we establish an explicit map** between a potential fluid and the shockwave metric perturbation, where the Einstein equation for the shockwave geometry is equivalen… ▽ More We study a correspondence between gravitational shockwave geometry and its fluid description near a Rindler horizon in Minkowski spacetime. Utilizing the Petrov classification that describes algebraic symmetries for Lorentzian spaces, we establish an explicit map** between a potential fluid and the shockwave metric perturbation, where the Einstein equation for the shockwave geometry is equivalent to the incompressibility condition of the fluid, augmented by a shockwave source. Then we consider an Ansatz of a stochastic quantum source for the potential fluid, which has the physical interpretation of shockwaves created by vacuum energy fluctuations. Under such circumstance, the Einstein equation, or equivalently, the incompressibility condition for the fluid, becomes a stochastic differential equation. By smearing the quantum source on a stretched horizon in a Lorentz invariant manner with a Planckian width (similarly to the membrane paradigm), we integrate fluctuations near the Rindler horizon to find an accumulated effect of the variance in the round-trip time of a photon traversing the horizon of a causal diamond. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Comments: 21 pages, 1 figure

Report number: CALT-TH 2024-016

Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects

Authors: Zicong Fan, Takehiko Ohkawa, Linlin Yang, Nie Lin, Zhishan Zhou, Shihao Zhou, Jiajun Liang, Zhong Gao, Xuanyang Zhang, Xue Zhang, Fei Li, Liu Zheng, Feng Lu, Karim Abou Zeid, Bastian Leibe, Jeongwan On, Seungryul Baek, Aditya Prakash, Saurabh Gupta, Kun He, Yoichi Sato, Otmar Hilliges, Hyung ** Chang, Angela Yao

Abstract: We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3D understanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D is challenging due to heavy occlusion, viewpoint bias, camera distortion, and motion blur from the… ▽ More We interact with the world with our hands and see it through our own (egocentric) perspective. A holistic 3D understanding of such interactions from egocentric views is important for tasks in robotics, AR/VR, action recognition and motion generation. Accurately reconstructing such interactions in 3D is challenging due to heavy occlusion, viewpoint bias, camera distortion, and motion blur from the head movement. To this end, we designed the HANDS23 challenge based on the AssemblyHands and ARCTIC datasets with carefully designed training and testing splits. Based on the results of the top submitted methods and more recent baselines on the leaderboards, we perform a thorough analysis on 3D hand(-object) reconstruction tasks. Our analysis demonstrates the effectiveness of addressing distortion specific to egocentric cameras, adopting high-capacity transformers to learn complex hand-object interactions, and fusing predictions from different views. Our study further reveals challenging scenarios intractable with state-of-the-art methods, such as fast hand motion, object reconstruction from narrow egocentric views, and close contact between two hands and objects. Our efforts will enrich the community's knowledge foundation and facilitate future hand studies on egocentric hand-object interactions. △ Less

Submitted 25 March, 2024; originally announced March 2024.

Exploiting Style Latent Flows for Generalizing Deepfake Video Detection

Authors: Jongwook Choi, Taehoon Kim, Yonghyun Jeong, Seungryul Baek, Jongwon Choi

Abstract: This paper presents a new approach for the detection of fake videos, based on the analysis of style latent vectors and their abnormal behavior in temporal changes in the generated videos. We discovered that the generated facial videos suffer from the temporal distinctiveness in the temporal changes of style latent vectors, which are inevitable during the generation of temporally stable videos with… ▽ More This paper presents a new approach for the detection of fake videos, based on the analysis of style latent vectors and their abnormal behavior in temporal changes in the generated videos. We discovered that the generated facial videos suffer from the temporal distinctiveness in the temporal changes of style latent vectors, which are inevitable during the generation of temporally stable videos with various facial expressions and geometric transformations. Our framework utilizes the StyleGRU module, trained by contrastive learning, to represent the dynamic properties of style latent vectors. Additionally, we introduce a style attention module that integrates StyleGRU-generated features with content-based features, enabling the detection of visual and temporal artifacts. We demonstrate our approach across various benchmark scenarios in deepfake detection, showing its superiority in cross-dataset and cross-manipulation scenarios. Through further analysis, we also validate the importance of using temporal changes of style latent vectors to improve the generality of deepfake video detection. △ Less

Submitted 20 May, 2024; v1 submitted 11 March, 2024; originally announced March 2024.

Comments: Preprint version, final version will be available at https://openaccess.thecvf.com The IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR) (2024) Published by: IEEE & CVF

VLM-PL: Advanced Pseudo Labeling Approach for Class Incremental Object Detection via Vision-Language Model

Authors: Junsu Kim, Yunhoe Ku, Jihyeon Kim, Junuk Cha, Seungryul Baek

Abstract: In the field of Class Incremental Object Detection (CIOD), creating models that can continuously learn like humans is a major challenge. Pseudo-labeling methods, although initially powerful, struggle with multi-scenario incremental learning due to their tendency to forget past knowledge. To overcome this, we introduce a new approach called Vision-Language Model assisted Pseudo-Labeling (VLM-PL). T… ▽ More In the field of Class Incremental Object Detection (CIOD), creating models that can continuously learn like humans is a major challenge. Pseudo-labeling methods, although initially powerful, struggle with multi-scenario incremental learning due to their tendency to forget past knowledge. To overcome this, we introduce a new approach called Vision-Language Model assisted Pseudo-Labeling (VLM-PL). This technique uses Vision-Language Model (VLM) to verify the correctness of pseudo ground-truths (GTs) without requiring additional model training. VLM-PL starts by deriving pseudo GTs from a pre-trained detector. Then, we generate custom queries for each pseudo GT using carefully designed prompt templates that combine image and text features. This allows the VLM to classify the correctness through its responses. Furthermore, VLM-PL integrates refined pseudo and real GTs from upcoming training, effectively combining new and old knowledge. Extensive experiments conducted on the Pascal VOC and MS COCO datasets not only highlight VLM-PL's exceptional performance in multi-scenario but also illuminate its effectiveness in dual-scenario by achieving state-of-the-art results in both. △ Less

Submitted 8 May, 2024; v1 submitted 8 March, 2024; originally announced March 2024.

Comments: Accept to CVPRW2024 (CLvision). The camera-ready version of the manuscript

SDDGR: Stable Diffusion-based Deep Generative Replay for Class Incremental Object Detection

Authors: Junsu Kim, Hoseong Cho, Jihyeon Kim, Yihalem Yimolal Tiruneh, Seungryul Baek

Abstract: In the field of class incremental learning (CIL), generative replay has become increasingly prominent as a method to mitigate the catastrophic forgetting, alongside the continuous improvements in generative models. However, its application in class incremental object detection (CIOD) has been significantly limited, primarily due to the complexities of scenes involving multiple labels. In this pape… ▽ More In the field of class incremental learning (CIL), generative replay has become increasingly prominent as a method to mitigate the catastrophic forgetting, alongside the continuous improvements in generative models. However, its application in class incremental object detection (CIOD) has been significantly limited, primarily due to the complexities of scenes involving multiple labels. In this paper, we propose a novel approach called stable diffusion deep generative replay (SDDGR) for CIOD. Our method utilizes a diffusion-based generative model with pre-trained text-to-diffusion networks to generate realistic and diverse synthetic images. SDDGR incorporates an iterative refinement strategy to produce high-quality images encompassing old classes. Additionally, we adopt an L2 knowledge distillation technique to improve the retention of prior knowledge in synthetic images. Furthermore, our approach includes pseudo-labeling for old objects within new task images, preventing misclassification as background elements. Extensive experiments on the COCO 2017 dataset demonstrate that SDDGR significantly outperforms existing algorithms, achieving a new state-of-the-art in various CIOD scenarios. The source code will be made available to the public. △ Less

Submitted 7 May, 2024; v1 submitted 27 February, 2024; originally announced February 2024.

Comments: Accept to CVPR 2024. The camera-ready version

Design and characterization of individual addressing optics based on multi-channel acousto-optic modulator for $^{171}$Yb$^+$ qubits

Authors: Sungjoo Lim, Seunghyun Baek, Jacob Whitlow, Marissa D'Onofrio, Tianyi Chen, Samuel Phiri, Stephen Crain, Kenneth R. Brown, Jungsang Kim, Junki Kim

Abstract: We present the design and characterization of individual addressing optics based on a multi-channel acousto-optic modulator (AOM) for trapped ytterbium-171 ions. The design parameters of the individual addressing system were determined based on the tradeoff between the expected crosstalk and the required numerical aperture of the projection objective lens. The target beam diameter and separation w… ▽ More We present the design and characterization of individual addressing optics based on a multi-channel acousto-optic modulator (AOM) for trapped ytterbium-171 ions. The design parameters of the individual addressing system were determined based on the tradeoff between the expected crosstalk and the required numerical aperture of the projection objective lens. The target beam diameter and separation were 1.90 $μ$m and 4.28 $μ$m, respectively. The individual beams shaped by the projection optics were characterized by an imaging sensor and a field probe ion. The resulting effective beam diameters and separations were approximately 2.34--2.36 $μ$m and 4.31 $μ$m, respectively, owing to residual aberration. △ Less

Submitted 30 March, 2024; v1 submitted 21 February, 2024; originally announced February 2024.

Comments: 14 pages, 5 figures

PARCv2: Physics-aware Recurrent Convolutional Neural Networks for Spatiotemporal Dynamics Modeling

Authors: Phong C. H. Nguyen, Xinlun Cheng, Shahab Azarfar, Pradeep Seshadri, Yen T. Nguyen, Munho Kim, Sanghun Choi, H. S. Udaykumar, Stephen Baek

Abstract: Modeling unsteady, fast transient, and advection-dominated physics problems is a pressing challenge for physics-aware deep learning (PADL). The physics of complex systems is governed by large systems of partial differential equations (PDEs) and ancillary constitutive models with nonlinear structures, as well as evolving state fields exhibiting sharp gradients and rapidly deforming material interfa… ▽ More Modeling unsteady, fast transient, and advection-dominated physics problems is a pressing challenge for physics-aware deep learning (PADL). The physics of complex systems is governed by large systems of partial differential equations (PDEs) and ancillary constitutive models with nonlinear structures, as well as evolving state fields exhibiting sharp gradients and rapidly deforming material interfaces. Here, we investigate an inductive bias approach that is versatile and generalizable to model generic nonlinear field evolution problems. Our study focuses on the recent physics-aware recurrent convolutions (PARC), which incorporates a differentiator-integrator architecture that inductively models the spatiotemporal dynamics of generic physical systems. We extend the capabilities of PARC to simulate unsteady, transient, and advection-dominant systems. The extended model, referred to as PARCv2, is equipped with differential operators to model advection-reaction-diffusion equations, as well as a hybrid integral solver for stable, long-time predictions. PARCv2 is tested on both standard benchmark problems in fluid dynamics, namely Burgers and Navier-Stokes equations, and then applied to more complex shock-induced reaction problems in energetic materials. We evaluate the behavior of PARCv2 in comparison to other physics-informed and learning bias models and demonstrate its potential to model unsteady and advection-dominant dynamics regimes. △ Less

Submitted 24 May, 2024; v1 submitted 19 February, 2024; originally announced February 2024.

Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once?

Authors: Gui** Son, Sangwon Baek, Sangdae Nam, Ilgyun Jeong, Seungone Kim

Abstract: Large language models (LLMs) are typically prompted to follow a single instruction per inference call. In this work, we analyze whether LLMs also hold the capability to handle multiple instructions simultaneously, denoted as Multi-Task Inference. For this purpose, we introduce the MTI Bench(Multi-Task Inference Benchmark), a comprehensive evaluation benchmark encompassing 5,000 instances across 25… ▽ More Large language models (LLMs) are typically prompted to follow a single instruction per inference call. In this work, we analyze whether LLMs also hold the capability to handle multiple instructions simultaneously, denoted as Multi-Task Inference. For this purpose, we introduce the MTI Bench(Multi-Task Inference Benchmark), a comprehensive evaluation benchmark encompassing 5,000 instances across 25 tasks. Each task in the MTI Bench involves 2 to 3 sub-tasks. As expected, we first demonstrate that Multi-Task Inference reduces the total inference time by 1.46 times in average since it does not require multiple inference calls. Interestingly, contrary to the expectation that LLMs would perform better when tasks are divided, we find that state-of-the-art LLMs, such as Llama-2-Chat-70B and GPT-4, show up to 7.3% and 12.4% improved performance with Multi-Task Inference compared to Single-Task Inference on the MTI Bench. We release the MTI Bench dataset and our code at this link https://github.com/gui**SON/MTI-Bench. △ Less

Submitted 6 June, 2024; v1 submitted 18 February, 2024; originally announced February 2024.

Comments: acl 2024 (main)

Hierarchical Position Embedding of Graphs with Landmarks and Clustering for Link Prediction

Authors: Minsang Kim, Seungjun Baek

Abstract: Learning positional information of nodes in a graph is important for link prediction tasks. We propose a representation of positional information using representative nodes called landmarks. A small number of nodes with high degree centrality are selected as landmarks, which serve as reference points for the nodes' positions. We justify this selection strategy for well-known random graph models an… ▽ More Learning positional information of nodes in a graph is important for link prediction tasks. We propose a representation of positional information using representative nodes called landmarks. A small number of nodes with high degree centrality are selected as landmarks, which serve as reference points for the nodes' positions. We justify this selection strategy for well-known random graph models and derive closed-form bounds on the average path lengths involving landmarks. In a model for power-law graphs, we prove that landmarks provide asymptotically exact information on inter-node distances. We apply theoretical insights to practical networks and propose Hierarchical Position embedding with Landmarks and Clustering (HPLC). HPLC combines landmark selection and graph clustering, where the graph is partitioned into densely connected clusters in which nodes with the highest degree are selected as landmarks. HPLC leverages the positional information of nodes based on landmarks at various levels of hierarchy such as nodes' distances to landmarks, inter-landmark distances and hierarchical grou** of clusters. Experiments show that HPLC achieves state-of-the-art performances of link prediction on various datasets in terms of HIT@K, MRR, and AUC. The code is available at \url{https://github.com/kmswin1/HPLC}. △ Less

Submitted 19 April, 2024; v1 submitted 12 February, 2024; originally announced February 2024.

Comments: The International World Wide Web Conference (WWW) 2024, Accepted paper

A Method for Decrypting Data Infected with Rhysida Ransomware

Authors: Giyoon Kim, Soo** Kang, Seungjun Baek, Kimoon Kim, Jongsung Kim

Abstract: Ransomware is malicious software that is a prominent global cybersecurity threat. Typically, ransomware encrypts data on a system, rendering the victim unable to decrypt it without the attacker's private key. Subsequently, victims often pay a substantial ransom to recover their data, yet some may still incur damage or loss. This study examines Rhysida ransomware, which caused significant damage in… ▽ More Ransomware is malicious software that is a prominent global cybersecurity threat. Typically, ransomware encrypts data on a system, rendering the victim unable to decrypt it without the attacker's private key. Subsequently, victims often pay a substantial ransom to recover their data, yet some may still incur damage or loss. This study examines Rhysida ransomware, which caused significant damage in the second half of 2023, and proposes a decryption method. Rhysida ransomware employed a secure random number generator to generate the encryption key and subsequently encrypt the data. However, an implementation vulnerability existed that enabled us to regenerate the internal state of the random number generator at the time of infection. We successfully decrypted the data using the regenerated random number generator. To the best of our knowledge, this is the first successful decryption of Rhysida ransomware. We aspire for our work to contribute to mitigating the damage inflicted by the Rhysida ransomware. △ Less

Submitted 9 February, 2024; originally announced February 2024.

Constrained Synthesis with Projected Diffusion Models

Authors: Jacob K Christopher, Stephen Baek, Ferdinando Fioretto

Abstract: This paper introduces an approach to endow generative diffusion processes the ability to satisfy and certify compliance with constraints and physical principles. The proposed method recast the traditional sampling process of generative diffusion models as a constrained optimization problem, steering the generated data distribution to remain within a specified region to ensure adherence to the give… ▽ More This paper introduces an approach to endow generative diffusion processes the ability to satisfy and certify compliance with constraints and physical principles. The proposed method recast the traditional sampling process of generative diffusion models as a constrained optimization problem, steering the generated data distribution to remain within a specified region to ensure adherence to the given constraints. These capabilities are validated on applications featuring both convex and challenging, non-convex, constraints as well as ordinary differential equations, in domains spanning from synthesizing new materials with precise morphometric properties, generating physics-informed motion, optimizing paths in planning scenarios, and human motion synthesis. △ Less

Submitted 23 May, 2024; v1 submitted 5 February, 2024; originally announced February 2024.

MolPLA: A Molecular Pretraining Framework for Learning Cores, R-Groups and their Linker Joints

Authors: Mogan Gim, Jueon Park, Soyon Park, Sanghoon Lee, Seungheun Baek, Junhyun Lee, Ngoc-Quang Nguyen, Jaewoo Kang

Abstract: Molecular core structures and R-groups are essential concepts in drug development. Integration of these concepts with conventional graph pre-training approaches can promote deeper understanding in molecules. We propose MolPLA, a novel pre-training framework that employs masked graph contrastive learning in understanding the underlying decomposable parts inmolecules that implicate their core struct… ▽ More Molecular core structures and R-groups are essential concepts in drug development. Integration of these concepts with conventional graph pre-training approaches can promote deeper understanding in molecules. We propose MolPLA, a novel pre-training framework that employs masked graph contrastive learning in understanding the underlying decomposable parts inmolecules that implicate their core structure and peripheral R-groups. Furthermore, we formulate an additional framework that grants MolPLA the ability to help chemists find replaceable R-groups in lead optimization scenarios. Experimental results on molecular property prediction show that MolPLA exhibits predictability comparable to current state-of-the-art models. Qualitative analysis implicate that MolPLA is capable of distinguishing core and R-group sub-structures, identifying decomposable regions in molecules and contributing to lead optimization scenarios by rationally suggesting R-group replacements given various query core templates. The code implementation for MolPLA and its pre-trained model checkpoint is available at https://github.com/dmis-lab/MolPLA △ Less

Submitted 30 January, 2024; originally announced January 2024.

Rapid Estimation of Left Ventricular Contractility with a Physics-Informed Neural Network Inverse Modeling Approach

Authors: Ehsan Naghavi, Haifeng Wang, Lei Fan, Jenny S. Choy, Ghassan Kassab, Seungik Baek, Lik-Chuan Lee

Abstract: Physics-based computer models based on numerical solution of the governing equations generally cannot make rapid predictions, which in turn, limits their applications in the clinic. To address this issue, we developed a physics-informed neural network (PINN) model that encodes the physics of a closed-loop blood circulation system embedding a left ventricle (LV). The PINN model is trained to satisf… ▽ More Physics-based computer models based on numerical solution of the governing equations generally cannot make rapid predictions, which in turn, limits their applications in the clinic. To address this issue, we developed a physics-informed neural network (PINN) model that encodes the physics of a closed-loop blood circulation system embedding a left ventricle (LV). The PINN model is trained to satisfy a system of ordinary differential equations (ODEs) associated with a lumped parameter description of the circulatory system. The model predictions have a maximum error of less than 5% when compared to those obtained by solving the ODEs numerically. An inverse modeling approach using the PINN model is also developed to rapidly estimate model parameters (in $\sim$ 3 mins) from single-beat LV pressure and volume waveforms. Using synthetic LV pressure and volume waveforms generated by the PINN model with different model parameter values, we show that the inverse modeling approach can recover the corresponding ground truth values, which suggests that the model parameters are unique. The PINN inverse modeling approach is then applied to estimate LV contractility indexed by the end-systolic elastance $E_{es}$ using waveforms acquired from 11 swine models, including waveforms acquired before and after administration of dobutamine (an inotropic agent) in 3 animals. The estimated $E_{es}$ is about 58% to 284% higher for the data associated with dobutamine compared to those without, which implies that this approach can be used to estimate LV contractility using single-beat measurements. The PINN inverse modeling can potentially be used in the clinic to simultaneously estimate LV contractility and other physiological parameters from single-beat measurements. △ Less

Submitted 14 January, 2024; originally announced January 2024.

3D Reconstruction of Interacting Multi-Person in Clothing from a Single Image

Authors: Junuk Cha, Hansol Lee, Jaewon Kim, Nhat Nguyen Bao Truong, Jae Shin Yoon, Seungryul Baek

Abstract: This paper introduces a novel pipeline to reconstruct the geometry of interacting multi-person in clothing on a globally coherent scene space from a single image. The main challenge arises from the occlusion: a part of a human body is not visible from a single view due to the occlusion by others or the self, which introduces missing geometry and physical implausibility (e.g., penetration). We over… ▽ More This paper introduces a novel pipeline to reconstruct the geometry of interacting multi-person in clothing on a globally coherent scene space from a single image. The main challenge arises from the occlusion: a part of a human body is not visible from a single view due to the occlusion by others or the self, which introduces missing geometry and physical implausibility (e.g., penetration). We overcome this challenge by utilizing two human priors for complete 3D geometry and surface contacts. For the geometry prior, an encoder learns to regress the image of a person with missing body parts to the latent vectors; a decoder decodes these vectors to produce 3D features of the associated geometry; and an implicit network combines these features with a surface normal map to reconstruct a complete and detailed 3D humans. For the contact prior, we develop an image-space contact detector that outputs a probability distribution of surface contacts between people in 3D. We use these priors to globally refine the body poses, enabling the penetration-free and accurate reconstruction of interacting multi-person in clothing on the scene space. The results demonstrate that our method is complete, globally coherent, and physically plausible compared to existing methods. △ Less

Submitted 2 April, 2024; v1 submitted 12 January, 2024; originally announced January 2024.

Comments: Accepted to WACV 2024

Low-Loss Polarization-Maintaining Optical Router for Photonic Quantum Information Processing

Authors: Pengfei Wang, Soyoung Baek, Keiichi Edamatsu, Fumihiro Kaneda

Abstract: In photonic quantum applications, optical routers are required to handle single photons with low loss, high speed, and preservation of their quantum states. Single-photon routing with maintained polarization states is particularly important for utilizing them as qubits. Here, we demonstrate a polarization-maintaining electro-optic router compatible with single photons. Our custom electro-optic mod… ▽ More In photonic quantum applications, optical routers are required to handle single photons with low loss, high speed, and preservation of their quantum states. Single-photon routing with maintained polarization states is particularly important for utilizing them as qubits. Here, we demonstrate a polarization-maintaining electro-optic router compatible with single photons. Our custom electro-optic modulator is embedded in a configuration of a Mach-Zehnder interferometer, where each optical component achieves polarization-maintaining operation. We observe the performance of the router with 2-4% loss, 20 dB switching extinction ratio, 2.9 ns rise time, and $>$ 99% polarization process fidelity to an ideal identity operation. △ Less

Submitted 6 April, 2024; v1 submitted 11 January, 2024; originally announced January 2024.

Journal ref: Jpn. J. Appl. Phys. 63 040901 (2024)

Limitations of Data-Driven Spectral Reconstruction -- Optics-Aware Analysis and Mitigation

Authors: Qiang Fu, Matheus Souza, Eunsue Choi, Suhyun Shin, Seung-Hwan Baek, Wolfgang Heidrich

Abstract: Hyperspectral imaging empowers machine vision systems with the distinct capability of identifying materials through recording their spectral signatures. Recent efforts in data-driven spectral reconstruction aim at extracting spectral information from RGB images captured by cost-effective RGB cameras, instead of dedicated hardware. In this paper we systematically analyze the performance of such m… ▽ More Hyperspectral imaging empowers machine vision systems with the distinct capability of identifying materials through recording their spectral signatures. Recent efforts in data-driven spectral reconstruction aim at extracting spectral information from RGB images captured by cost-effective RGB cameras, instead of dedicated hardware. In this paper we systematically analyze the performance of such methods, evaluating both the practical limitations with respect to current datasets and overfitting, as well as fundamental limitations with respect to the nature of the information encoded in the RGB images, and the dependency of this information on the optical system of the camera. We find that, the current models are not robust under slight variations, e.g., in noise level or compression of the RGB file. Without modeling underrepresented spectral content, existing datasets and the models trained on them are limited in their ability to cope with challenging metameric colors. To mitigate this issue, we propose to exploit the combination of metameric data augmentation and optical lens aberrations to improve the encoding of the metameric information into the RGB image, which paves the road towards higher performing spectral imaging and reconstruction approaches. △ Less

Submitted 2 April, 2024; v1 submitted 8 January, 2024; originally announced January 2024.

Comments: 12 pages, 7 figures, 8 tables

UGPNet: Universal Generative Prior for Image Restoration

Authors: Hwayoon Lee, Kyoungkook Kang, Hyeongmin Lee, Seung-Hwan Baek, Sunghyun Cho

Abstract: Recent image restoration methods can be broadly categorized into two classes: (1) regression methods that recover the rough structure of the original image without synthesizing high-frequency details and (2) generative methods that synthesize perceptually-realistic high-frequency details even though the resulting image deviates from the original structure of the input. While both directions have b… ▽ More Recent image restoration methods can be broadly categorized into two classes: (1) regression methods that recover the rough structure of the original image without synthesizing high-frequency details and (2) generative methods that synthesize perceptually-realistic high-frequency details even though the resulting image deviates from the original structure of the input. While both directions have been extensively studied in isolation, merging their benefits with a single framework has been rarely studied. In this paper, we propose UGPNet, a universal image restoration framework that can effectively achieve the benefits of both approaches by simply adopting a pair of an existing regression model and a generative model. UGPNet first restores the image structure of a degraded input using a regression model and synthesizes a perceptually-realistic image with a generative model on top of the regressed output. UGPNet then combines the regressed output and the synthesized output, resulting in a final result that faithfully reconstructs the structure of the original image in addition to perceptually-realistic textures. Our extensive experiments on deblurring, denoising, and super-resolution demonstrate that UGPNet can successfully exploit both regression and generative methods for high-fidelity image restoration. △ Less

Submitted 30 December, 2023; originally announced January 2024.

Comments: Accepted to WACV 2024

Quantum-Gravitational Null Raychaudhuri Equation

Authors: Sang-Eon Bak, Maulik Parikh, Sudipta Sarkar, Francesco Setti

Abstract: We consider a congruence of null geodesics in the presence of a quantized spacetime metric. The coupling to a quantum metric induces fluctuations in the congruence; we calculate the change in the area of a pencil of geodesics induced by such fluctuations. For the gravitational field in its vacuum state, we find that quantum gravity contributes a correction to the null Raychaudhuri equation which i… ▽ More We consider a congruence of null geodesics in the presence of a quantized spacetime metric. The coupling to a quantum metric induces fluctuations in the congruence; we calculate the change in the area of a pencil of geodesics induced by such fluctuations. For the gravitational field in its vacuum state, we find that quantum gravity contributes a correction to the null Raychaudhuri equation which is of the same sign as the classical terms. We thus derive a quantum-gravitational focusing theorem valid for linearized quantum gravity. △ Less

Submitted 28 December, 2023; originally announced December 2023.

Comments: 14 pages, 1 figure, LaTeX

Dynamic Appearance Modeling of Clothed 3D Human Avatars using a Single Camera

Authors: Hansol Lee, Junuk Cha, Yunhoe Ku, Jae Shin Yoon, Seungryul Baek

Abstract: The appearance of a human in clothing is driven not only by the pose but also by its temporal context, i.e., motion. However, such context has been largely neglected by existing monocular human modeling methods whose neural networks often struggle to learn a video of a person with large dynamics due to the motion ambiguity, i.e., there exist numerous geometric configurations of clothes that are de… ▽ More The appearance of a human in clothing is driven not only by the pose but also by its temporal context, i.e., motion. However, such context has been largely neglected by existing monocular human modeling methods whose neural networks often struggle to learn a video of a person with large dynamics due to the motion ambiguity, i.e., there exist numerous geometric configurations of clothes that are dependent on the context of motion even for the same pose. In this paper, we introduce a method for high-quality modeling of clothed 3D human avatars using a video of a person with dynamic movements. The main challenge comes from the lack of 3D ground truth data of geometry and its temporal correspondences. We address this challenge by introducing a novel compositional human modeling framework that takes advantage of both explicit and implicit human modeling. For explicit modeling, a neural network learns to generate point-wise shape residuals and appearance features of a 3D body model by comparing its 2D rendering results and the original images. This explicit model allows for the reconstruction of discriminative 3D motion features from UV space by encoding their temporal correspondences. For implicit modeling, an implicit network combines the appearance and 3D motion features to decode high-fidelity clothed 3D human avatars with motion-dependent geometry and texture. The experiments show that our method can generate a large variation of secondary motion in a physically plausible way. △ Less

Submitted 28 December, 2023; originally announced December 2023.

The Fourth International Verification of Neural Networks Competition (VNN-COMP 2023): Summary and Results

Authors: Christopher Brix, Stanley Bak, Changliu Liu, Taylor T. Johnson

Abstract: This report summarizes the 4th International Verification of Neural Networks Competition (VNN-COMP 2023), held as a part of the 6th Workshop on Formal Methods for ML-Enabled Autonomous Systems (FoMLAS), that was collocated with the 35th International Conference on Computer-Aided Verification (CAV). VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural… ▽ More This report summarizes the 4th International Verification of Neural Networks Competition (VNN-COMP 2023), held as a part of the 6th Workshop on Formal Methods for ML-Enabled Autonomous Systems (FoMLAS), that was collocated with the 35th International Conference on Computer-Aided Verification (CAV). VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools, encourage the standardization of tool interfaces, and bring together the neural network verification community. To this end, standardized formats for networks (ONNX) and specification (VNN-LIB) were defined, tools were evaluated on equal-cost hardware (using an automatic evaluation pipeline based on AWS instances), and tool parameters were chosen by the participants before the final test sets were made public. In the 2023 iteration, 7 teams participated on a diverse set of 10 scored and 4 unscored benchmarks. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition. △ Less

Submitted 27 December, 2023; originally announced December 2023.

Comments: arXiv admin note: text overlap with arXiv:2212.10376

ParamISP: Learned Forward and Inverse ISPs using Camera Parameters

Authors: Woohyeok Kim, Geonu Kim, Junyong Lee, Seungyong Lee, Seung-Hwan Baek, Sunghyun Cho

Abstract: RAW images are rarely shared mainly due to its excessive data size compared to their sRGB counterparts obtained by camera ISPs. Learning the forward and inverse processes of camera ISPs has been recently demonstrated, enabling physically-meaningful RAW-level image processing on input sRGB images. However, existing learning-based ISP methods fail to handle the large variations in the ISP processes… ▽ More RAW images are rarely shared mainly due to its excessive data size compared to their sRGB counterparts obtained by camera ISPs. Learning the forward and inverse processes of camera ISPs has been recently demonstrated, enabling physically-meaningful RAW-level image processing on input sRGB images. However, existing learning-based ISP methods fail to handle the large variations in the ISP processes with respect to camera parameters such as ISO and exposure time, and have limitations when used for various applications. In this paper, we propose ParamISP, a learning-based method for forward and inverse conversion between sRGB and RAW images, that adopts a novel neural-network module to utilize camera parameters, which is dubbed as ParamNet. Given the camera parameters provided in the EXIF data, ParamNet converts them into a feature vector to control the ISP networks. Extensive experiments demonstrate that ParamISP achieve superior RAW and sRGB reconstruction results compared to previous methods and it can be effectively used for a variety of applications such as deblurring dataset synthesis, raw deblurring, HDR reconstruction, and camera-to-camera transfer. △ Less

Submitted 14 April, 2024; v1 submitted 20 December, 2023; originally announced December 2023.

Class-Wise Buffer Management for Incremental Object Detection: An Effective Buffer Training Strategy

Authors: Junsu Kim, Sumin Hong, Chanwoo Kim, Jihyeon Kim, Yihalem Yimolal Tiruneh, Jeongwan On, Jihyun Song, Sunhwa Choi, Seungryul Baek

Abstract: Class incremental learning aims to solve a problem that arises when continuously adding unseen class instances to an existing model This approach has been extensively studied in the context of image classification; however its applicability to object detection is not well established yet. Existing frameworks using replay methods mainly collect replay data without considering the model being traine… ▽ More Class incremental learning aims to solve a problem that arises when continuously adding unseen class instances to an existing model This approach has been extensively studied in the context of image classification; however its applicability to object detection is not well established yet. Existing frameworks using replay methods mainly collect replay data without considering the model being trained and tend to rely on randomness or the number of labels of each sample. Also, despite the effectiveness of the replay, it was not yet optimized for the object detection task. In this paper, we introduce an effective buffer training strategy (eBTS) that creates the optimized replay buffer on object detection. Our approach incorporates guarantee minimum and hierarchical sampling to establish the buffer customized to the trained model. %These methods can facilitate effective retrieval of prior knowledge. Furthermore, we use the circular experience replay training to optimally utilize the accumulated buffer data. Experiments on the MS COCO dataset demonstrate that our eBTS achieves state-of-the-art performance compared to the existing replay schemes. △ Less

Submitted 14 December, 2023; originally announced December 2023.

Comments: 5 pages, 3 figures, Accepted at ICASSP 2024

Differentiable Point-based Inverse Rendering

Authors: Hoon-Gyu Chung, Seokjun Choi, Seung-Hwan Baek

Abstract: We present differentiable point-based inverse rendering, DPIR, an analysis-by-synthesis method that processes images captured under diverse illuminations to estimate shape and spatially-varying BRDF. To this end, we adopt point-based rendering, eliminating the need for multiple samplings per ray, typical of volumetric rendering, thus significantly enhancing the speed of inverse rendering. To reali… ▽ More We present differentiable point-based inverse rendering, DPIR, an analysis-by-synthesis method that processes images captured under diverse illuminations to estimate shape and spatially-varying BRDF. To this end, we adopt point-based rendering, eliminating the need for multiple samplings per ray, typical of volumetric rendering, thus significantly enhancing the speed of inverse rendering. To realize this idea, we devise a hybrid point-volumetric representation for geometry and a regularized basis-BRDF representation for reflectance. The hybrid geometric representation enables fast rendering through point-based splatting while retaining the geometric details and stability inherent to SDF-based representations. The regularized basis-BRDF mitigates the ill-posedness of inverse rendering stemming from limited light-view angular samples. We also propose an efficient shadow detection method using point-based shadow map rendering. Our extensive evaluations demonstrate that DPIR outperforms prior works in terms of reconstruction accuracy, computational efficiency, and memory footprint. Furthermore, our explicit point-based representation and rendering enables intuitive geometry and reflectance editing. △ Less

Submitted 25 March, 2024; v1 submitted 4 December, 2023; originally announced December 2023.

Coverage Explorer: Coverage-guided Test Generation for Cyber Physical Systems

Authors: Sanaz Sheikhi, Stanley Bak

Abstract: Given the safety-critical functions of autonomous cyber-physical systems (CPS) across diverse domains, testing these systems is essential. While conventional software and hardware testing methodologies offer partial insights, they frequently do not provide adequate coverage in a CPS. In this study, we introduce a testing framework designed to systematically formulate test cases, effectively explor… ▽ More Given the safety-critical functions of autonomous cyber-physical systems (CPS) across diverse domains, testing these systems is essential. While conventional software and hardware testing methodologies offer partial insights, they frequently do not provide adequate coverage in a CPS. In this study, we introduce a testing framework designed to systematically formulate test cases, effectively exploring the state space of CPS. This framework introduces a coverage-centric sampling technique, coupled with a cluster-based methodology for training a surrogate model. The framework then uses model predictive control within the surrogate model to generates test cases tailored to CPS specifications. To evaluate the efficacy of the framework, we applied it on several benchmarks, spanning from a kinematic car to systems like an unmanned aircraft collision avoidance system (ACAS XU) and automatic transmission system. Comparative analyses were conducted against alternative test generation strategies, including randomized testing, as well as falsification using S-TaLiRo. △ Less

Submitted 4 December, 2023; originally announced December 2023.

Dispersed Structured Light for Hyperspectral 3D Imaging

Authors: Suhyun Shin, Seokjun Choi, Felix Heide, Seung-Hwan Baek

Abstract: Hyperspectral 3D imaging aims to acquire both depth and spectral information of a scene. However, existing methods are either prohibitively expensive and bulky or compromise on spectral and depth accuracy. In this work, we present Dispersed Structured Light (DSL), a cost-effective and compact method for accurate hyperspectral 3D imaging. DSL modifies a traditional projector-camera system by placin… ▽ More Hyperspectral 3D imaging aims to acquire both depth and spectral information of a scene. However, existing methods are either prohibitively expensive and bulky or compromise on spectral and depth accuracy. In this work, we present Dispersed Structured Light (DSL), a cost-effective and compact method for accurate hyperspectral 3D imaging. DSL modifies a traditional projector-camera system by placing a sub-millimeter thick diffraction grating film front of the projector. The grating disperses structured light based on light wavelength. To utilize the dispersed structured light, we devise a model for dispersive projection image formation and a per-pixel hyperspectral 3D reconstruction method. We validate DSL by instantiating a compact experimental prototype. DSL achieves spectral accuracy of 18.8nm full-width half-maximum (FWHM) and depth error of 1mm. We demonstrate that DSL outperforms prior work on practical hyperspectral 3D imaging. DSL promises accurate and practical hyperspectral 3D imaging for diverse application domains, including computer vision and graphics, cultural heritage, geology, and biology. △ Less

Submitted 25 March, 2024; v1 submitted 30 November, 2023; originally announced November 2023.

Spectral and Polarization Vision: Spectro-polarimetric Real-world Dataset

Authors: Yu** Jeon, Eunsue Choi, Youngchan Kim, Yunseong Moon, Khalid Omer, Felix Heide, Seung-Hwan Baek

Abstract: Image datasets are essential not only in validating existing methods in computer vision but also in develo** new methods. Most existing image datasets focus on trichromatic intensity images to mimic human vision. However, polarization and spectrum, the wave properties of light that animals in harsh environments and with limited brain capacity often rely on, remain underrepresented in existing da… ▽ More Image datasets are essential not only in validating existing methods in computer vision but also in develo** new methods. Most existing image datasets focus on trichromatic intensity images to mimic human vision. However, polarization and spectrum, the wave properties of light that animals in harsh environments and with limited brain capacity often rely on, remain underrepresented in existing datasets. Although spectro-polarimetric datasets exist, these datasets have insufficient object diversity, limited illumination conditions, linear-only polarization data, and inadequate image count. Here, we introduce two spectro-polarimetric datasets: trichromatic Stokes images and hyperspectral Stokes images. These novel datasets encompass both linear and circular polarization; they introduce multiple spectral channels; and they feature a broad selection of real-world scenes. With our dataset in hand, we analyze the spectro-polarimetric image statistics, develop efficient representations of such high-dimensional data, and evaluate spectral dependency of shape-from-polarization methods. As such, the proposed dataset promises a foundation for data-driven spectro-polarimetric imaging and vision research. Dataset and code will be publicly available. △ Less

Submitted 30 November, 2023; v1 submitted 29 November, 2023; originally announced November 2023.

3D Teeth Reconstruction from Panoramic Radiographs using Neural Implicit Functions

Authors: Sihwa Park, Seongjun Kim, In-Seok Song, Seung Jun Baek

Abstract: Panoramic radiography is a widely used imaging modality in dental practice and research. However, it only provides flattened 2D images, which limits the detailed assessment of dental structures. In this paper, we propose Occudent, a framework for 3D teeth reconstruction from panoramic radiographs using neural implicit functions, which, to the best of our knowledge, is the first work to do so. For… ▽ More Panoramic radiography is a widely used imaging modality in dental practice and research. However, it only provides flattened 2D images, which limits the detailed assessment of dental structures. In this paper, we propose Occudent, a framework for 3D teeth reconstruction from panoramic radiographs using neural implicit functions, which, to the best of our knowledge, is the first work to do so. For a given point in 3D space, the implicit function estimates whether the point is occupied by a tooth, and thus implicitly determines the boundaries of 3D tooth shapes. Firstly, Occudent applies multi-label segmentation to the input panoramic radiograph. Next, tooth shape embeddings as well as tooth class embeddings are generated from the segmentation outputs, which are fed to the reconstruction network. A novel module called Conditional eXcitation (CX) is proposed in order to effectively incorporate the combined shape and class embeddings into the implicit function. The performance of Occudent is evaluated using both quantitative and qualitative measures. Importantly, Occudent is trained and validated with actual panoramic radiographs as input, distinct from recent works which used synthesized images. Experiments demonstrate the superiority of Occudent over state-of-the-art methods. △ Less

Submitted 28 November, 2023; originally announced November 2023.

Comments: 12 pages, 2 figures, accepted to International Conference on Medical Image Computing and Computer-Assisted Intervention MICCAI 2023

Clinical Characteristics and Laboratory Biomarkers in ICU-admitted Septic Patients with and without Bacteremia

Authors: Sangwon Baek, Seung Jun Lee

Abstract: Few studies have investigated the diagnostic utilities of biomarkers for predicting bacteremia among septic patients admitted to intensive care units (ICU). Therefore, this study evaluated the prediction power of laboratory biomarkers to utilize those markers with high performance to optimize the predictive model for bacteremia. This retrospective cross-sectional study was conducted at the ICU dep… ▽ More Few studies have investigated the diagnostic utilities of biomarkers for predicting bacteremia among septic patients admitted to intensive care units (ICU). Therefore, this study evaluated the prediction power of laboratory biomarkers to utilize those markers with high performance to optimize the predictive model for bacteremia. This retrospective cross-sectional study was conducted at the ICU department of Gyeongsang National University Changwon Hospital in 2019. Adult patients qualifying SEPSIS-3 (increase in sequential organ failure score greater than or equal to 2) criteria with at least two sets of blood culture were selected. Collected data was initially analyzed independently to identify the significant predictors, which was then used to build the multivariable logistic regression (MLR) model. A total of 218 patients with 48 cases of true bacteremia were analyzed in this research. Both CRP and PCT showed a substantial area under the curve (AUC) value for discriminating bacteremia among septic patients (0.757 and 0.845, respectively). To further enhance the predictive accuracy, we combined PCT, bilirubin, neutrophil lymphocyte ratio (NLR), platelets, lactic acid, erythrocyte sedimentation rate (ESR), and Glasgow Coma Scale (GCS) score to build the predictive model with an AUC of 0.907 (95% CI, 0.843 to 0.956). In addition, a high association between bacteremia and mortality rate was discovered through the survival analysis (0.004). While PCT is certainly a useful index for distinguishing patients with and without bacteremia by itself, our MLR model indicates that the accuracy of bacteremia prediction substantially improves by the combined use of PCT, bilirubin, NLR, platelets, lactic acid, ESR, and GCS score. △ Less

Submitted 16 November, 2023; v1 submitted 14 November, 2023; originally announced November 2023.

Comments: This article is not the right fit to be published as preprint in arXiv

Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization

Authors: Jangwhan Lee, Minsoo Kim, Seungcheol Baek, Seok Joong Hwang, Wonyong Sung, Jungwook Choi

Abstract: Large Language Models (LLMs) are proficient in natural language processing tasks, but their deployment is often restricted by extensive parameter sizes and computational demands. This paper focuses on post-training quantization (PTQ) in LLMs, specifically 4-bit weight and 8-bit activation (W4A8) quantization, to enhance computational efficiency -- a topic less explored compared to weight-only quan… ▽ More Large Language Models (LLMs) are proficient in natural language processing tasks, but their deployment is often restricted by extensive parameter sizes and computational demands. This paper focuses on post-training quantization (PTQ) in LLMs, specifically 4-bit weight and 8-bit activation (W4A8) quantization, to enhance computational efficiency -- a topic less explored compared to weight-only quantization. We present two innovative techniques: activation-quantization-aware scaling (AQAS) and sequence-length-aware calibration (SLAC) to enhance PTQ by considering the combined effects on weights and activations and aligning calibration sequence lengths to target tasks. Moreover, we introduce dINT, a hybrid data format combining integer and denormal representations, to address the underflow issue in W4A8 quantization, where small values are rounded to zero. Through rigorous evaluations of LLMs, including OPT and LLaMA, we demonstrate that our techniques significantly boost task accuracies to levels comparable with full-precision models. By develo** arithmetic units compatible with dINT, we further confirm that our methods yield a 2$\times$ hardware efficiency improvement compared to 8-bit integer MAC unit. △ Less

Submitted 9 November, 2023; originally announced November 2023.

Comments: EMNLP 2023 Main Conference

Toward Reinforcement Learning-based Rectilinear Macro Placement Under Human Constraints

Authors: Tuyen P. Le, Hieu T. Nguyen, Seungyeol Baek, Taeyoun Kim, Jungwoo Lee, Seongjung Kim, Hyun** Kim, Misu Jung, Daehoon Kim, Seokyong Lee, Daewoo Choi

Abstract: Macro placement is a critical phase in chip design, which becomes more intricate when involving general rectilinear macros and layout areas. Furthermore, macro placement that incorporates human-like constraints, such as design hierarchy and peripheral bias, has the potential to significantly reduce the amount of additional manual labor required from designers. This study proposes a methodology tha… ▽ More Macro placement is a critical phase in chip design, which becomes more intricate when involving general rectilinear macros and layout areas. Furthermore, macro placement that incorporates human-like constraints, such as design hierarchy and peripheral bias, has the potential to significantly reduce the amount of additional manual labor required from designers. This study proposes a methodology that leverages an approach suggested by Google's Circuit Training (G-CT) to provide a learning-based macro placer that not only supports placing rectilinear cases, but also adheres to crucial human-like design principles. Our experimental results demonstrate the effectiveness of our framework in achieving power-performance-area (PPA) metrics and in obtaining placements of high quality, comparable to those produced with human intervention. Additionally, our methodology shows potential as a generalized model to address diverse macro shapes and layout areas. △ Less

Submitted 2 November, 2023; originally announced November 2023.

Comments: Fast ML for Science @ ICCAD 2023

Blending Bathymetry: Combination of image-derived parametric approximations and celerity data sets for nearshore bathymetry estimation

Authors: Jonghyun Lee, Katherine DeVore, Tyler Hesser, A. Spicer Bak, Katherine Brodie, Brittany Bruder, Matthew Farthing

Abstract: Estimation of nearshore bathymetry is important for accurate prediction of nearshore wave conditions. However, direct data collection is expensive and time-consuming while accurate airborne lidar-based survey is limited by breaking waves and decreased light penetration affected by water turbidity. Instead, tower-based platforms or Unmanned Aircraft System (UAS) can provide indirect video-based obs… ▽ More Estimation of nearshore bathymetry is important for accurate prediction of nearshore wave conditions. However, direct data collection is expensive and time-consuming while accurate airborne lidar-based survey is limited by breaking waves and decreased light penetration affected by water turbidity. Instead, tower-based platforms or Unmanned Aircraft System (UAS) can provide indirect video-based observations. The video-based time-series imagery provides wave celerity information and time-averaged (timex) or variance enhanced (var) images identify persistent regions of wave breaking. In this work, we propose a rapid and improved bathymetry estimation method that takes advantage of image-derived wave celerity and a first-order bathymetry estimate from Parameter Beach Tool (PBT), software that fits parameterized sandbar and slope forms to the timex or var images. Two different sources of the data, PBT and wave celerity, are combined or blended optimally based on their assumed accuracy in a statistical framework. The PBT-derived bathymetry serves as "prior" coarse-scale background information and then is updated and corrected with the imagery-derived wave data through the dispersion relationship, which results in a better bathymetry estimate that is consistent with imagery-based wave data. To illustrate the accuracy of our proposed method, imagery data sets collected in 2017 at the US Army EDRC's Field Research Facility in Duck, NC under different weather and wave height conditions are tested. Estimated bathymetry profiles are remarkably close to the direct survey data. The computational time for the estimation from PBT-based bathymetry and imagery-derived wave celerity is only about five minutes on a free Google Cloud node with one CPU core. These promising results indicate the feasibility of reliable real-time bathymetry imaging during a single flight of UAS. △ Less

Submitted 2 November, 2023; originally announced November 2023.

Comments: 21 pages, 14 figures, preprints

Error-disturbance uncertainty relations in a superconducting quantum processor

Authors: Tingrui Dong, Soyoung Baek, Fumihiro Kaneda, Keiichi Edamatsu

Abstract: We experimentally test the error-disturbance uncertainty relation (EDR) in generalized, variable strength measurements of superconducting qubits on a NISQ processor. Making use of sequential weak measurements that keeps the initial signal state practically unchanged prior to the main measurement, we demonstrate that the Heisenberg EDR is violated, yet the Ozawa and Branciard EDRs are valid through… ▽ More We experimentally test the error-disturbance uncertainty relation (EDR) in generalized, variable strength measurements of superconducting qubits on a NISQ processor. Making use of sequential weak measurements that keeps the initial signal state practically unchanged prior to the main measurement, we demonstrate that the Heisenberg EDR is violated, yet the Ozawa and Branciard EDRs are valid throughout the range of measurement strengths from no measurement to projection measurement. Our results verify that universal EDRs are valid even in a noisy quantum processor and will stimulate research on measurement-based quantum information and communication protocols using a NISQ processor. △ Less

Submitted 1 November, 2023; originally announced November 2023.

Comments: 6 pages, 4 figures. arXiv admin note: text overlap with arXiv:1308.5868

Finite-size scaling analysis of the two-dimensional random transverse-field Ising ferromagnet

Authors: Jiwon Choi, Seung Ki Baek

Abstract: The random transverse-field Ising ferromagnet (RTFIF) is a highly disordered quantum system which contains randomness in the coupling strengths as well as in the transverse-field strengths. In one dimension, the critical properties are governed by an infinite-randomness fixed point (IRFP), and renormalization-group studies argue that the two-dimensional (2D) model is also governed by an IRFP. Howe… ▽ More The random transverse-field Ising ferromagnet (RTFIF) is a highly disordered quantum system which contains randomness in the coupling strengths as well as in the transverse-field strengths. In one dimension, the critical properties are governed by an infinite-randomness fixed point (IRFP), and renormalization-group studies argue that the two-dimensional (2D) model is also governed by an IRFP. However, even the location of the critical point remains unsettled among quantum Monte Carlo (QMC) studies. In this work, we perform extensive QMC simulations to locate the quantum critical point and attempt a finite-size scaling analysis to observe the critical behavior. We estimate the critical field strength of the 2D RTFIF as $Γ_c = 7.52(2)$, together with critical exponents such as $β=1.5(3)$, $ν= 1.6(3)$, and $z=3.3(3)$ or $ψ=0.50(3)$. We have also considered the McCoy-Wu model, which has randomness in the ferromagnetic coupling strengths but not in the transverse-field strength. Our QMC calculation shows that the critical behavior of the 2D McCoy-Wu model is closer to that of the 2D transverse-field Ising spin glass than to that of the 2D RTFIF. These numerical findings enhance our understanding of disordered 2D quantum systems. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Journal ref: Physical Review B 108, 144204 (2023)

Atomic Insights into the Oxidative Degradation Mechanisms of Sulfide Solid Electrolytes

Authors: Chuntian Cao, Matthew R. Carbone, Cem Komurcuoglu, Jagriti S. Shekhawat, Kerry Sun, Haoyue Guo, Sizhan Liu, Ke Chen, Seong-Min Bak, Yonghua Du, Conan Weiland, Xiao Tong, Dan Steingart, Shinjae Yoo, Nongnuch Artrith, Alexander Urban, Deyu Lu, Feng Wang

Abstract: Electrochemical degradation of solid electrolytes is a major roadblock in the development of solid-state batteries, and the formed solid-solid interphase (SSI) plays a key role in the performance of solid-state batteries. In this study, by combining experimental X-ray absorption spectroscopy (XAS) measurements, first-principles simulations, and unsupervised machine learning, we have unraveled the… ▽ More Electrochemical degradation of solid electrolytes is a major roadblock in the development of solid-state batteries, and the formed solid-solid interphase (SSI) plays a key role in the performance of solid-state batteries. In this study, by combining experimental X-ray absorption spectroscopy (XAS) measurements, first-principles simulations, and unsupervised machine learning, we have unraveled the atomic-scale oxidative degradation mechanisms of sulfide electrolytes at the interface using the baseline Li3PS4 (LPS) electrolyte as a model system. The degradation begins with a decrease of Li neighbor affinity to S atoms upon initial delithiation, followed by the formation of S-S bonds as the PS4 tetrahedron deforms. After the first delithiation cycle, the PS4 motifs become strongly distorted and PS3 motifs start to form. Spectral fingerprints of the local structural evolution are identified, which correspond to the main peak broadening and the peak shifting to a higher energy by about 2.5 eV in P K-edge XAS and a new peak emerging at 2473 eV in S K-edge XAS during delithiation. The spectral fingerprints serve as a proxy for the electrochemical stability of phosphorus sulfide solid electrolytes beyond LPS, as demonstrated in argyrodite Li6PS5Cl. We observed that the strong distortion and destruction of PS4 tetrahedra and the formation of S-S bonds are correlated with an increased interfacial impedance. To the best of our knowledge, this study showcases the first atomic-scale insights into the oxidative degradation mechanism of the LPS electrolyte, which can provide guidance for controlling macroscopic reactions through microstructural engineering and, more generally, can advance the rational design of sulfide electrolytes. △ Less

Submitted 1 October, 2023; originally announced October 2023.