Showing 1–50 of 69 results for author: Kwon, B

Structure of singularities for the Euler-Poisson system of ion dynamics

Authors: Junsik Bae, Yunjoo Kim, Bongsuk Kwon

Abstract: We study the formation of singularity for the isothermal Euler-Poisson system arising from plasma physics. Contrast to the previous studies yielding only limited information on the blow-up solutions, for instance, sufficient conditions for the blow-up and the temporal blow-up rate along the characteristic curve, we rather give a constructive proof of singularity formation from smooth initial data.… ▽ More We study the formation of singularity for the isothermal Euler-Poisson system arising from plasma physics. Contrast to the previous studies yielding only limited information on the blow-up solutions, for instance, sufficient conditions for the blow-up and the temporal blow-up rate along the characteristic curve, we rather give a constructive proof of singularity formation from smooth initial data. More specifically, employing the stable blow-up profile of the Burgers equation in the self-similar variables, we establish the global stability estimate in the self-similar time, which yields the asymptotic behavior of blow-up solutions near the singularity point. Our analysis indicates that the smooth solution to the Euler-Poisson system can develop a cusp-type singularity; it exhibits $C^1$ blow-up in a finite time, while it belongs to $C^{1/3}$ at the blow-up time, provided that smooth initial data are sufficiently close to the blow-up profile in some weighted $C^4$-topology. We also present a similar result for the isentropic case, and discuss noteworthy differences in the analysis. △ Less

Submitted 4 May, 2024; originally announced May 2024.

Comments: 57 pages

MiMICRI: Towards Domain-centered Counterfactual Explanations of Cardiovascular Image Classification Models

Authors: Grace Guo, Lifu Deng, Animesh Tandon, Alex Endert, Bum Chul Kwon

Abstract: The recent prevalence of publicly accessible, large medical imaging datasets has led to a proliferation of artificial intelligence (AI) models for cardiovascular image classification and analysis. At the same time, the potentially significant impacts of these models have motivated the development of a range of explainable AI (XAI) methods that aim to explain model predictions given certain image i… ▽ More The recent prevalence of publicly accessible, large medical imaging datasets has led to a proliferation of artificial intelligence (AI) models for cardiovascular image classification and analysis. At the same time, the potentially significant impacts of these models have motivated the development of a range of explainable AI (XAI) methods that aim to explain model predictions given certain image inputs. However, many of these methods are not developed or evaluated with domain experts, and explanations are not contextualized in terms of medical expertise or domain knowledge. In this paper, we propose a novel framework and python library, MiMICRI, that provides domain-centered counterfactual explanations of cardiovascular image classification models. MiMICRI helps users interactively select and replace segments of medical images that correspond to morphological structures. From the counterfactuals generated, users can then assess the influence of each segment on model predictions, and validate the model against known medical facts. We evaluate this library with two medical experts. Our evaluation demonstrates that a domain-centered XAI approach can enhance the interpretability of model explanations, and help experts reason about models in terms of relevant domain knowledge. However, concerns were also surfaced about the clinical plausibility of the counterfactuals generated. We conclude with a discussion on the generalizability and trustworthiness of the MiMICRI framework, as well as the implications of our findings on the development of domain-centered XAI methods for model interpretability in healthcare contexts. △ Less

Submitted 24 April, 2024; originally announced April 2024.

Comments: 14 pages, 6 figures, ACM FAccT 2024

ASAP: Interpretable Analysis and Summarization of AI-generated Image Patterns at Scale

Authors: **bin Huang, Chen Chen, Aditi Mishra, Bum Chul Kwon, Zhicheng Liu, Chris Bryan

Abstract: Generative image models have emerged as a promising technology to produce realistic images. Despite potential benefits, concerns grow about its misuse, particularly in generating deceptive images that could raise significant ethical, legal, and societal issues. Consequently, there is growing demand to empower users to effectively discern and comprehend patterns of AI-generated images. To this end,… ▽ More Generative image models have emerged as a promising technology to produce realistic images. Despite potential benefits, concerns grow about its misuse, particularly in generating deceptive images that could raise significant ethical, legal, and societal issues. Consequently, there is growing demand to empower users to effectively discern and comprehend patterns of AI-generated images. To this end, we developed ASAP, an interactive visualization system that automatically extracts distinct patterns of AI-generated images and allows users to interactively explore them via various views. To uncover fake patterns, ASAP introduces a novel image encoder, adapted from CLIP, which transforms images into compact "distilled" representations, enriched with information for differentiating authentic and fake images. These representations generate gradients that propagate back to the attention maps of CLIP's transformer block. This process quantifies the relative importance of each pixel to image authenticity or fakeness, exposing key deceptive patterns. ASAP enables the at scale interactive analysis of these patterns through multiple, coordinated visualizations. This includes a representation overview with innovative cell glyphs to aid in the exploration and qualitative evaluation of fake patterns across a vast array of images, as well as a pattern view that displays authenticity-indicating patterns in images and quantifies their impact. ASAP supports the analysis of cutting-edge generative models with the latest architectures, including GAN-based models like proGAN and diffusion models like the latent diffusion model. We demonstrate ASAP's usefulness through two usage scenarios using multiple fake image detection benchmark datasets, revealing its ability to identify and understand hidden patterns in AI-generated images, especially in detecting fake human faces produced by diffusion-based techniques. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Comments: 9 pages, 6 figures

HyperCLOVA X Technical Report

Authors: Kang Min Yoo, Jaegeun Han, Sookyo In, Heewon Jeon, Jisu Jeong, Jaewook Kang, Hyunwook Kim, Kyung-Min Kim, Munhyong Kim, Sungju Kim, Donghyun Kwak, Hanock Kwak, Se Jung Kwon, Bado Lee, Dongsoo Lee, Gichang Lee, Jooho Lee, Baeseong Park, Seong** Shin, Joonsang Yu, Seolki Baek, Sumin Byeon, Eungsup Cho, Dooseok Choe, Jeesung Han , et al. (371 additional authors not shown)

Abstract: We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment t… ▽ More We introduce HyperCLOVA X, a family of large language models (LLMs) tailored to the Korean language and culture, along with competitive capabilities in English, math, and coding. HyperCLOVA X was trained on a balanced mix of Korean, English, and code data, followed by instruction-tuning with high-quality human-annotated datasets while abiding by strict safety guidelines reflecting our commitment to responsible AI. The model is evaluated across various benchmarks, including comprehensive reasoning, knowledge, commonsense, factuality, coding, math, chatting, instruction-following, and harmlessness, in both Korean and English. HyperCLOVA X exhibits strong reasoning capabilities in Korean backed by a deep understanding of the language and cultural nuances. Further analysis of the inherent bilingual nature and its extension to multilingualism highlights the model's cross-lingual proficiency and strong generalization ability to untargeted languages, including machine translation between several language pairs and cross-lingual inference tasks. We believe that HyperCLOVA X can provide helpful guidance for regions or countries in develo** their sovereign LLMs. △ Less

Submitted 13 April, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Comments: 44 pages; updated authors list and fixed author names

A 28.6 mJ/iter Stable Diffusion Processor for Text-to-Image Generation with Patch Similarity-based Sparsity Augmentation and Text-based Mixed-Precision

Authors: Jiwon Choi, Wooyoung Jo, Seongyon Hong, Beomseok Kwon, Wonhoon Park, Hoi-Jun Yoo

Abstract: This paper presents an energy-efficient stable diffusion processor for text-to-image generation. While stable diffusion attained attention for high-quality image synthesis results, its inherent characteristics hinder its deployment on mobile platforms. The proposed processor achieves high throughput and energy efficiency with three key features as solutions: 1) Patch similarity-based sparsity augm… ▽ More This paper presents an energy-efficient stable diffusion processor for text-to-image generation. While stable diffusion attained attention for high-quality image synthesis results, its inherent characteristics hinder its deployment on mobile platforms. The proposed processor achieves high throughput and energy efficiency with three key features as solutions: 1) Patch similarity-based sparsity augmentation (PSSA) to reduce external memory access (EMA) energy of self-attention score by 60.3 %, leading to 37.8 % total EMA energy reduction. 2) Text-based important pixel spotting (TIPS) to allow 44.8 % of the FFN layer workload to be processed with low-precision activation. 3) Dual-mode bit-slice core (DBSC) architecture to enhance energy efficiency in FFN layers by 43.0 %. The proposed processor is implemented in 28 nm CMOS technology and achieves 3.84 TOPS peak throughput with 225.6 mW average power consumption. In sum, 28.6 mJ/iteration highly energy-efficient text-to-image generation processor can be achieved at MS-COCO dataset. △ Less

Submitted 14 March, 2024; v1 submitted 7 March, 2024; originally announced March 2024.

Comments: Accepted at 2024 IEEE International Symposium on Circuits and Systems (ISCAS)

No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization

Authors: June Yong Yang, Byeongwook Kim, Jeongin Bae, Beomseok Kwon, Gunho Park, Eunho Yang, Se Jung Kwon, Dongsoo Lee

Abstract: Key-Value (KV) Caching has become an essential technique for accelerating the inference speed and throughput of generative Large Language Models~(LLMs). However, the memory footprint of the KV cache poses a critical bottleneck in LLM deployment as the cache size grows with batch size and sequence length, often surpassing even the size of the model itself. Although recent methods were proposed to s… ▽ More Key-Value (KV) Caching has become an essential technique for accelerating the inference speed and throughput of generative Large Language Models~(LLMs). However, the memory footprint of the KV cache poses a critical bottleneck in LLM deployment as the cache size grows with batch size and sequence length, often surpassing even the size of the model itself. Although recent methods were proposed to select and evict unimportant KV pairs from the cache to reduce memory consumption, the potential ramifications of eviction on the generative process are yet to be thoroughly examined. In this paper, we examine the detrimental impact of cache eviction and observe that unforeseen risks arise as the information contained in the KV pairs is exhaustively discarded, resulting in safety breaches, hallucinations, and context loss. Surprisingly, we find that preserving even a small amount of information contained in the evicted KV pairs via reduced precision quantization substantially recovers the incurred degradation. On the other hand, we observe that the important KV pairs must be kept at a relatively higher precision to safeguard the generation quality. Motivated by these observations, we propose \textit{Mixed-precision KV cache}~(MiKV), a reliable cache compression method that simultaneously preserves the context details by retaining the evicted KV pairs in low-precision and ensure generation quality by kee** the important KV pairs in high-precision. Experiments on diverse benchmarks and LLM backbones show that our proposed method offers a state-of-the-art trade-off between compression ratio and performance, compared to other baselines. △ Less

Submitted 28 February, 2024; originally announced February 2024.

Shall We Team Up: Exploring Spontaneous Cooperation of Competing LLM Agents

Authors: Zengqing Wu, Run Peng, Shuyuan Zheng, Qianying Liu, Xu Han, Brian Inhyuk Kwon, Makoto Onizuka, Shaojie Tang, Chuan Xiao

Abstract: Large Language Models (LLMs) have increasingly been utilized in social simulations, where they are often guided by carefully crafted instructions to stably exhibit human-like behaviors during simulations. Nevertheless, we doubt the necessity of sha** agents' behaviors for accurate social simulations. Instead, this paper emphasizes the importance of spontaneous phenomena, wherein agents deeply en… ▽ More Large Language Models (LLMs) have increasingly been utilized in social simulations, where they are often guided by carefully crafted instructions to stably exhibit human-like behaviors during simulations. Nevertheless, we doubt the necessity of sha** agents' behaviors for accurate social simulations. Instead, this paper emphasizes the importance of spontaneous phenomena, wherein agents deeply engage in contexts and make adaptive decisions without explicit directions. We explored spontaneous cooperation across three competitive scenarios and successfully simulated the gradual emergence of cooperation, findings that align closely with human behavioral data. This approach not only aids the computational social science community in bridging the gap between simulations and real-world dynamics but also offers the AI community a novel method to assess LLMs' capability of deliberate reasoning. △ Less

Submitted 2 July, 2024; v1 submitted 19 February, 2024; originally announced February 2024.

Comments: Source codes available at https://github.com/wuzengqing001225/SABM_ShallWeTeamUp

Approximate solutions for the Vlasov--Poisson system with boundary layers

Authors: Chang-Yeol Jung, Bongsuk Kwon, Masahiro Suzuki, Masahiro Takayama

Abstract: We construct the approximate solutions to the Vlasov--Poisson system in a half-space, which arises in the study of the quasi-neutral limit problem in the presence of a sharp boundary layer, referred as to the plasma sheath in the context of plasma physics. The quasi-neutrality is an important characteristic of plasmas and its scale is characterized by a small parameter, called the Debye length.… ▽ More We construct the approximate solutions to the Vlasov--Poisson system in a half-space, which arises in the study of the quasi-neutral limit problem in the presence of a sharp boundary layer, referred as to the plasma sheath in the context of plasma physics. The quasi-neutrality is an important characteristic of plasmas and its scale is characterized by a small parameter, called the Debye length. We present the approximate equations obtained by a formal expansion in the parameter and study the properties of the approximate solutions. Moreover, we present numerical experiments demonstrating that the approximate solutions converge to those of the Vlasov--Poisson system as the parameter goes to zero. △ Less

Submitted 12 January, 2024; originally announced January 2024.

From-Ground-To-Objects: Coarse-to-Fine Self-supervised Monocular Depth Estimation of Dynamic Objects with Ground Contact Prior

Authors: Jaeho Moon, Juan Luis Gonzalez Bello, Byeongjun Kwon, Munchurl Kim

Abstract: Self-supervised monocular depth estimation (DE) is an approach to learning depth without costly depth ground truths. However, it often struggles with moving objects that violate the static scene assumption during training. To address this issue, we introduce a coarse-to-fine training strategy leveraging the ground contacting prior based on the observation that most moving objects in outdoor scenes… ▽ More Self-supervised monocular depth estimation (DE) is an approach to learning depth without costly depth ground truths. However, it often struggles with moving objects that violate the static scene assumption during training. To address this issue, we introduce a coarse-to-fine training strategy leveraging the ground contacting prior based on the observation that most moving objects in outdoor scenes contact the ground. In the coarse training stage, we exclude the objects in dynamic classes from the reprojection loss calculation to avoid inaccurate depth learning. To provide precise supervision on the depth of the objects, we present a novel Ground-contacting-prior Disparity Smoothness Loss (GDS-Loss) that encourages a DE network to align the depth of the objects with their ground-contacting points. Subsequently, in the fine training stage, we refine the DE network to learn the detailed depth of the objects from the reprojection loss, while ensuring accurate DE on the moving object regions by employing our regularization loss with a cost-volume-based weighting factor. Our overall coarse-to-fine training strategy can easily be integrated with existing DE methods without any modifications, significantly enhancing DE performance on challenging Cityscapes and KITTI datasets, especially in the moving object regions. △ Less

Submitted 15 December, 2023; originally announced December 2023.

Latent Space Explorer: Visual Analytics for Multimodal Latent Space Exploration

Authors: Bum Chul Kwon, Samuel Friedman, Kai Xu, Steven A Lubitz, Anthony Philippakis, Puneet Batra, Patrick T Ellinor, Kenney Ng

Abstract: Machine learning models built on training data with multiple modalities can reveal new insights that are not accessible through unimodal datasets. For example, cardiac magnetic resonance images (MRIs) and electrocardiograms (ECGs) are both known to capture useful information about subjects' cardiovascular health status. A multimodal machine learning model trained from large datasets can potentiall… ▽ More Machine learning models built on training data with multiple modalities can reveal new insights that are not accessible through unimodal datasets. For example, cardiac magnetic resonance images (MRIs) and electrocardiograms (ECGs) are both known to capture useful information about subjects' cardiovascular health status. A multimodal machine learning model trained from large datasets can potentially predict the onset of heart-related diseases and provide novel medical insights about the cardiovascular system. Despite the potential benefits, it is difficult for medical experts to explore multimodal representation models without visual aids and to test the predictive performance of the models on various subpopulations. To address the challenges, we developed a visual analytics system called Latent Space Explorer. Latent Space Explorer provides interactive visualizations that enable users to explore the multimodal representation of subjects, define subgroups of interest, interactively decode data with different modalities with the selected subjects, and inspect the accuracy of the embedding in downstream prediction tasks. A user study was conducted with medical experts and their feedback provided useful insights into how Latent Space Explorer can help their analysis and possible new direction for further development in the medical domain. △ Less

Submitted 1 December, 2023; originally announced December 2023.

Comments: 7 pages, 5 figures

Sample Dominance Aware Framework via Non-Parametric Estimation for Spontaneous Brain-Computer Interface

Authors: Byeong-Hoo Lee, Byoung-Hee Kwon, Seong-Whan Lee

Abstract: Deep learning has shown promise in decoding brain signals, such as electroencephalogram (EEG), in the field of brain-computer interfaces (BCIs). However, the non-stationary characteristics of EEG signals pose challenges for training neural networks to acquire appropriate knowledge. Inconsistent EEG signals resulting from these non-stationary characteristics can lead to poor performance. Therefore,… ▽ More Deep learning has shown promise in decoding brain signals, such as electroencephalogram (EEG), in the field of brain-computer interfaces (BCIs). However, the non-stationary characteristics of EEG signals pose challenges for training neural networks to acquire appropriate knowledge. Inconsistent EEG signals resulting from these non-stationary characteristics can lead to poor performance. Therefore, it is crucial to investigate and address sample inconsistency to ensure robust performance in spontaneous BCIs. In this study, we introduce the concept of sample dominance as a measure of EEG signal inconsistency and propose a method to modulate its effect on network training. We present a two-stage dominance score estimation technique that compensates for performance degradation caused by sample inconsistencies. Our proposed method utilizes non-parametric estimation to infer sample inconsistency and assigns each sample a dominance score. This score is then aggregated with the loss function during training to modulate the impact of sample inconsistency. Furthermore, we design a curriculum learning approach that gradually increases the influence of inconsistent signals during training to improve overall performance. We evaluate our proposed method using public spontaneous BCI dataset. The experimental results confirm that our findings highlight the importance of addressing sample dominance for achieving robust performance in spontaneous BCIs. △ Less

Submitted 14 November, 2023; v1 submitted 13 November, 2023; originally announced November 2023.

Comments: 5 pages, 2 figures

Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models

Authors: Jung Hwan Heo, Jeonghoon Kim, Beomseok Kwon, Byeongwook Kim, Se Jung Kwon, Dongsoo Lee

Abstract: Large Language Models (LLMs) have recently demonstrated remarkable success across various tasks. However, efficiently serving LLMs has been a challenge due to the large memory bottleneck, specifically in small batch inference settings (e.g. mobile devices). Weight-only quantization can be a promising approach, but sub-4 bit quantization remains a challenge due to large-magnitude activation outlier… ▽ More Large Language Models (LLMs) have recently demonstrated remarkable success across various tasks. However, efficiently serving LLMs has been a challenge due to the large memory bottleneck, specifically in small batch inference settings (e.g. mobile devices). Weight-only quantization can be a promising approach, but sub-4 bit quantization remains a challenge due to large-magnitude activation outliers. To mitigate the undesirable outlier effect, we first propose per-IC quantization, a simple yet effective method that creates quantization groups within each input channel (IC) rather than the conventional per-output-channel (per-OC). Our method is motivated by the observation that activation outliers affect the input dimension of the weight matrix, so similarly grou** the weights in the IC direction can isolate outliers within a group. We also find that activation outliers do not dictate quantization difficulty, and inherent weight sensitivities also exist. With per-IC quantization as a new outlier-friendly scheme, we propose Adaptive Dimensions (AdaDim), a versatile quantization framework that can adapt to various weight sensitivity patterns. We demonstrate the effectiveness of AdaDim by augmenting prior methods such as Round-To-Nearest and GPTQ, showing significant improvements across various language modeling benchmarks for both base (up to +4.7% on MMLU) and instruction-tuned (up to +10% on HumanEval) LLMs. Code is available at https://github.com/johnheo/adadim-llm △ Less

Submitted 24 March, 2024; v1 submitted 27 September, 2023; originally announced September 2023.

Comments: ICLR 2024. 19 pages, 11 figures, 10 tables

People's Perceptions Toward Bias and Related Concepts in Large Language Models: A Systematic Review

Authors: Lu Wang, Max Song, Rezvaneh Rezapour, Bum Chul Kwon, **a Huh-Yoo

Abstract: Large language models (LLMs) have brought breakthroughs in tasks including translation, summarization, information retrieval, and language generation, gaining growing interest in the CHI community. Meanwhile, the literature shows researchers' controversial perceptions about the efficacy, ethics, and intellectual abilities of LLMs. However, we do not know how people perceive LLMs that are pervasive… ▽ More Large language models (LLMs) have brought breakthroughs in tasks including translation, summarization, information retrieval, and language generation, gaining growing interest in the CHI community. Meanwhile, the literature shows researchers' controversial perceptions about the efficacy, ethics, and intellectual abilities of LLMs. However, we do not know how people perceive LLMs that are pervasive in everyday tools, specifically regarding their experience with LLMs around bias, stereotypes, social norms, or safety. In this study, we conducted a systematic review to understand what empirical insights papers have gathered about people's perceptions toward LLMs. From a total of 231 retrieved papers, we full-text reviewed 15 papers that recruited human evaluators to assess their experiences with LLMs. We report different biases and related concepts investigated by these studies, four broader LLM application areas, the evaluators' perceptions toward LLMs' performances including advantages, biases, and conflicting perceptions, factors influencing these perceptions, and concerns about LLM applications. △ Less

Submitted 2 March, 2024; v1 submitted 25 September, 2023; originally announced September 2023.

Towards Visualization Thumbnail Designs that Entice Reading Data-driven Articles

Authors: Hwiyeon Kim, Joohee Kim, Yunha Han, Hwajung Hong, Oh-Sang Kwon, Young-Woo Park, Niklas Elmqvist, Sungahn Ko, Bum Chul Kwon

Abstract: As online news increasingly include data journalism, there is a corresponding increase in the incorporation of visualization in article thumbnail images. However, little research exists on the design rationale for visualization thumbnails, such as resizing, crop**, simplifying, and embellishing charts that appear within the body of the associated article. Therefore, in this paper we aim to under… ▽ More As online news increasingly include data journalism, there is a corresponding increase in the incorporation of visualization in article thumbnail images. However, little research exists on the design rationale for visualization thumbnails, such as resizing, crop**, simplifying, and embellishing charts that appear within the body of the associated article. Therefore, in this paper we aim to understand these design choices and determine what makes a visualization thumbnail inviting and interpretable. To this end, we first survey visualization thumbnails collected online and discuss visualization thumbnail practices with data journalists and news graphics designers. Based on the survey and discussion results, we then define a design space for visualization thumbnails and conduct a user study with four types of visualization thumbnails derived from the design space. The study results indicate that different chart components play different roles in attracting reader attention and enhancing reader understandability of the visualization thumbnails. We also find various thumbnail design strategies for effectively combining the charts' components, such as a data summary with highlights and data labels, and a visual legend with text labels and Human Recognizable Objects (HROs), into thumbnails. Ultimately, we distill our findings into design implications that allow effective visualization thumbnail designs for data-rich news articles. Our work can thus be seen as a first step toward providing structured guidance on how to design compelling thumbnails for data stories. △ Less

Submitted 26 May, 2023; originally announced May 2023.

Comments: To appear in IEEE Transactions on Visualization and Computer Graphics, 16 pages, 6 figures, 5 tables. arXiv admin note: text overlap with arXiv:1908.06922

Finspector: A Human-Centered Visual Inspection Tool for Exploring and Comparing Biases among Foundation Models

Authors: Bum Chul Kwon, Nandana Mihindukulasooriya

Abstract: Pre-trained transformer-based language models are becoming increasingly popular due to their exceptional performance on various benchmarks. However, concerns persist regarding the presence of hidden biases within these models, which can lead to discriminatory outcomes and reinforce harmful stereotypes. To address this issue, we propose Finspector, a human-centered visual inspection tool designed t… ▽ More Pre-trained transformer-based language models are becoming increasingly popular due to their exceptional performance on various benchmarks. However, concerns persist regarding the presence of hidden biases within these models, which can lead to discriminatory outcomes and reinforce harmful stereotypes. To address this issue, we propose Finspector, a human-centered visual inspection tool designed to detect biases in different categories through log-likelihood scores generated by language models. The goal of the tool is to enable researchers to easily identify potential biases using visual analytics, ultimately contributing to a fairer and more just deployment of these models in both academic and industrial settings. Finspector is available at https://github.com/IBM/finspector. △ Less

Submitted 26 May, 2023; originally announced May 2023.

Comments: ACL 2023 System Demonstrations, 9 pages, 3 figures

PromptAid: Prompt Exploration, Perturbation, Testing and Iteration using Visual Analytics for Large Language Models

Authors: Aditi Mishra, Utkarsh Soni, Anjana Arunkumar, **bin Huang, Bum Chul Kwon, Chris Bryan

Abstract: Large Language Models (LLMs) have gained widespread popularity due to their ability to perform ad-hoc Natural Language Processing (NLP) tasks with a simple natural language prompt. Part of the appeal for LLMs is their approachability to the general public, including individuals with no prior technical experience in NLP techniques. However, natural language prompts can vary significantly in terms o… ▽ More Large Language Models (LLMs) have gained widespread popularity due to their ability to perform ad-hoc Natural Language Processing (NLP) tasks with a simple natural language prompt. Part of the appeal for LLMs is their approachability to the general public, including individuals with no prior technical experience in NLP techniques. However, natural language prompts can vary significantly in terms of their linguistic structure, context, and other semantics. Modifying one or more of these aspects can result in significant differences in task performance. Non-expert users may find it challenging to identify the changes needed to improve a prompt, especially when they lack domain-specific knowledge and lack appropriate feedback. To address this challenge, we present PromptAid, a visual analytics system designed to interactively create, refine, and test prompts through exploration, perturbation, testing, and iteration. PromptAid uses multiple, coordinated visualizations which allow users to improve prompts by using the three strategies: keyword perturbations, paraphrasing perturbations, and obtaining the best set of in-context few-shot examples. PromptAid was designed through an iterative prototy** process involving NLP experts and was evaluated through quantitative and qualitative assessments for LLMs. Our findings indicate that PromptAid helps users to iterate over prompt template alterations with less cognitive overhead, generate diverse prompts with help of recommendations, and analyze the performance of the generated prompts while surpassing existing state-of-the-art prompting interfaces in performance. △ Less

Submitted 8 April, 2023; v1 submitted 4 April, 2023; originally announced April 2023.

Normal forms for rational 3-tangles

Authors: Bo-hyun Kwon, Jung Hoon Lee

Abstract: In this paper, we define the \textit{normal form} of collections of disjoint three \textit{bridge arcs} for a given rational $3$-tangle. We show that there is a sequence of \textit{normal jump moves} which leads one to the other for two normal forms of the same rational 3-tangle. In this paper, we define the \textit{normal form} of collections of disjoint three \textit{bridge arcs} for a given rational $3$-tangle. We show that there is a sequence of \textit{normal jump moves} which leads one to the other for two normal forms of the same rational 3-tangle. △ Less

Submitted 13 March, 2023; originally announced March 2023.

Comments: 10 pages, 11 figures

MSC Class: 57M10

On detecting the trivial rational $3$-tangle

Authors: Bo-hyun Kwon

Abstract: An important issue in classifying the rational $3$-tangle is how to know whether or not the given tangle is the trivial rational 3-tangle called $\infty$-tangle. The author\cite{1} provided a certain algorithm to detect the $\infty$-tangle. In this paper, we give a much simpler method to detect the $\infty$-tangle by using the $\textit{bridge arc replacement}$. We hope that this method can help pr… ▽ More An important issue in classifying the rational $3$-tangle is how to know whether or not the given tangle is the trivial rational 3-tangle called $\infty$-tangle. The author\cite{1} provided a certain algorithm to detect the $\infty$-tangle. In this paper, we give a much simpler method to detect the $\infty$-tangle by using the $\textit{bridge arc replacement}$. We hope that this method can help prove many application problems such as the classification of $3$-bridge knots. △ Less

Submitted 13 March, 2023; originally announced March 2023.

MSC Class: 57K10

Causalvis: Visualizations for Causal Inference

Authors: Grace Guo, Ehud Karavani, Alex Endert, Bum Chul Kwon

Abstract: Causal inference is a statistical paradigm for quantifying causal effects using observational data. It is a complex process, requiring multiple steps, iterations, and collaborations with domain experts. Analysts often rely on visualizations to evaluate the accuracy of each step. However, existing visualization toolkits are not designed to support the entire causal inference process within computat… ▽ More Causal inference is a statistical paradigm for quantifying causal effects using observational data. It is a complex process, requiring multiple steps, iterations, and collaborations with domain experts. Analysts often rely on visualizations to evaluate the accuracy of each step. However, existing visualization toolkits are not designed to support the entire causal inference process within computational environments familiar to analysts. In this paper, we address this gap with Causalvis, a Python visualization package for causal inference. Working closely with causal inference experts, we adopted an iterative design process to develop four interactive visualization modules to support causal inference analysis tasks. The modules are then presented back to the experts for feedback and evaluation. We found that Causalvis effectively supported the iterative causal inference process. We discuss the implications of our findings for designing visualizations for causal inference, particularly for tasks of communication and collaboration. △ Less

Submitted 1 March, 2023; originally announced March 2023.

Comments: 20 pages, 14 figures

Automatic Network Adaptation for Ultra-Low Uniform-Precision Quantization

Authors: Seongmin Park, Beomseok Kwon, Jieun Lim, Kyuyoung Sim, Tae-Ho Kim, Jungwook Choi

Abstract: Uniform-precision neural network quantization has gained popularity since it simplifies densely packed arithmetic unit for high computing capability. However, it ignores heterogeneous sensitivity to the impact of quantization errors across the layers, resulting in sub-optimal inference accuracy. This work proposes a novel neural architecture search called neural channel expansion that adjusts the… ▽ More Uniform-precision neural network quantization has gained popularity since it simplifies densely packed arithmetic unit for high computing capability. However, it ignores heterogeneous sensitivity to the impact of quantization errors across the layers, resulting in sub-optimal inference accuracy. This work proposes a novel neural architecture search called neural channel expansion that adjusts the network structure to alleviate accuracy degradation from ultra-low uniform-precision quantization. The proposed method selectively expands channels for the quantization sensitive layers while satisfying hardware constraints (e.g., FLOPs, PARAMs). Based on in-depth analysis and experiments, we demonstrate that the proposed method can adapt several popular networks channels to achieve superior 2-bit quantization accuracy on CIFAR10 and ImageNet. In particular, we achieve the best-to-date Top-1/Top-5 accuracy for 2-bit ResNet50 with smaller FLOPs and the parameter size. △ Less

Submitted 29 March, 2023; v1 submitted 21 December, 2022; originally announced December 2022.

Comments: Accepted as a full paper by the TinyML Research Symposium 2023

Hybrid Paradigm-based Brain-Computer Interface for Robotic Arm Control

Authors: Byeong-Hoo Lee, Jeong-Hyun Cho, Byung-Hee Kwon

Abstract: Brain-computer interface (BCI) uses brain signals to communicate with external devices without actual control. Particularly, BCI is one of the interfaces for controlling the robotic arm. In this study, we propose a knowledge distillation-based framework to manipulate robotic arm through hybrid paradigm induced EEG signals for practical use. The teacher model is designed to decode input data hierar… ▽ More Brain-computer interface (BCI) uses brain signals to communicate with external devices without actual control. Particularly, BCI is one of the interfaces for controlling the robotic arm. In this study, we propose a knowledge distillation-based framework to manipulate robotic arm through hybrid paradigm induced EEG signals for practical use. The teacher model is designed to decode input data hierarchically and transfer knowledge to student model. To this end, soft labels and distillation loss functions are applied to the student model training. According to experimental results, student model achieved the best performance among the singular architecture-based methods. It is confirmed that using hierarchical models and knowledge distillation, the performance of a simple architecture can be improved. Since it is uncertain what knowledge is transferred, it is important to clarify this part in future studies. △ Less

Submitted 14 December, 2022; originally announced December 2022.

Decoding Multi-class Motor-related Intentions with User-optimized and Robust BCI System Based on Multimodal Dataset

Authors: Jeong-Hyun Cho, Byoung-Hee Kwon, Byeong-Hoo Lee

Abstract: A brain-computer interface (BCI) based on electroencephalography (EEG) can be useful for rehabilitation and the control of external devices. Five gras** tasks were decoded for motor execution (ME) and motor imagery (MI). During this experiment, eight healthy subjects were asked to imagine and grasp five objects. Analysis of EEG signals was performed after detecting muscle signals on electromyogr… ▽ More A brain-computer interface (BCI) based on electroencephalography (EEG) can be useful for rehabilitation and the control of external devices. Five gras** tasks were decoded for motor execution (ME) and motor imagery (MI). During this experiment, eight healthy subjects were asked to imagine and grasp five objects. Analysis of EEG signals was performed after detecting muscle signals on electromyograms (EMG) with a time interval selection technique on data taken from these ME and MI experiments. By refining only data corresponding to the exact time when the users performed the motor intention, the proposed method can train the decoding model using only the EEG data generated by various motor intentions with strong correlation with a specific class. There was an accuracy of 70.73% for ME and 47.95% for MI for the five offline tasks. This method may be applied to future applications, such as controlling robot hands with BCIs. △ Less

Submitted 14 December, 2022; originally announced December 2022.

Comments: Submitted to 2023 11th IEEE International Winter Conference on Brain-Computer Interface

Target-centered Subject Transfer Framework for EEG Data Augmentation

Authors: Kang Yin, Byeong-Hoo Lee, Byoung-Hee Kwon, Jeong-Hyun Cho

Abstract: Data augmentation approaches are widely explored for the enhancement of decoding electroencephalogram signals. In subject-independent brain-computer interface system, domain adaption and generalization are utilized to shift source subjects' data distribution to match the target subject as an augmentation. However, previous works either introduce noises (e.g., by noise addition or generation with r… ▽ More Data augmentation approaches are widely explored for the enhancement of decoding electroencephalogram signals. In subject-independent brain-computer interface system, domain adaption and generalization are utilized to shift source subjects' data distribution to match the target subject as an augmentation. However, previous works either introduce noises (e.g., by noise addition or generation with random noises) or modify target data, thus, cannot well depict the target data distribution and hinder further analysis. In this paper, we propose a target-centered subject transfer framework as a data augmentation approach. A subset of source data is first constructed to maximize the source-target relevance. Then, the generative model is applied to transfer the data to target domain. The proposed framework enriches the explainability of target domain by adding extra real data, instead of noises. It shows superior performance compared with other data augmentation methods. Extensive experiments are conducted to verify the effectiveness and robustness of our approach as a prosperous tool for further research. △ Less

Submitted 23 November, 2022; originally announced December 2022.

Channel Optimized Visual Imagery based Robotic Arm Control under the Online Environment

Authors: Byoung-Hee Kwon, Byeong-Hoo Lee, Jeong-Hyun Cho

Abstract: An electroencephalogram is an effective approach that provides a bidirectional pathway between the user and computer in a non-invasive way. In this study, we adopted the visual imagery data for controlling the BCI-based robotic arm. Visual imagery increases the power of the alpha frequency range of the visual cortex over time as the user performs the task. We proposed a deep learning architecture… ▽ More An electroencephalogram is an effective approach that provides a bidirectional pathway between the user and computer in a non-invasive way. In this study, we adopted the visual imagery data for controlling the BCI-based robotic arm. Visual imagery increases the power of the alpha frequency range of the visual cortex over time as the user performs the task. We proposed a deep learning architecture to decode the visual imagery data using only two channels and also we investigated the combination of two EEG channels that has significant classification performance. When using the proposed method, the highest classification performance using two channels in the offline experiment was 0.661. Also, the highest success rate in the online experiment using two channels (AF3-Oz) was 0.78. Our results provide the possibility of controlling the BCI-based robotic arm using visual imagery data. △ Less

Submitted 23 November, 2022; originally announced November 2022.

Comments: 4 pages, 2 figures, 3 tables

Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022 challenge: Report

Authors: Andrey Ignatov, Radu Timofte, ** Zhang, Feng Zhang, Gaocheng Yu, Zhe Ma, Hongbin Wang, Minsu Kwon, Haotian Qian, Wentao Tong, Pan Mu, Zi** Wang, Guang**g Yan, Brian Lee, Lei Fei, Huai** Chen, Hyebin Cho, Byeongjun Kwon, Munchurl Kim, Mingyang Qian, Huixin Ma, Yanan Li, Xiaotao Wang, Lei Lei

Abstract: As mobile cameras with compact optics are unable to produce a strong bokeh effect, lots of interest is now devoted to deep learning-based solutions for this task. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based bokeh effect rendering approach that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale EBB!… ▽ More As mobile cameras with compact optics are unable to produce a strong bokeh effect, lots of interest is now devoted to deep learning-based solutions for this task. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based bokeh effect rendering approach that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale EBB! bokeh dataset consisting of 5K shallow / wide depth-of-field image pairs captured using the Canon 7D DSLR camera. The runtime of the resulting models was evaluated on the Kirin 9000's Mali GPU that provides excellent acceleration results for the majority of common deep learning ops. A detailed description of all models developed in this challenge is provided in this paper. △ Less

Submitted 7 November, 2022; originally announced November 2022.

Comments: arXiv admin note: substantial text overlap with arXiv:2211.03885; text overlap with arXiv:2105.07809, arXiv:2211.04470, arXiv:2211.05256, arXiv:2211.05910

RMExplorer: A Visual Analytics Approach to Explore the Performance and the Fairness of Disease Risk Models on Population Subgroups

Authors: Bum Chul Kwon, Uri Kartoun, Shaan Khurshid, Mikhail Yurochkin, Subha Maity, Deanna G Brockman, Amit V Khera, Patrick T Ellinor, Steven A Lubitz, Kenney Ng

Abstract: Disease risk models can identify high-risk patients and help clinicians provide more personalized care. However, risk models developed on one dataset may not generalize across diverse subpopulations of patients in different datasets and may have unexpected performance. It is challenging for clinical researchers to inspect risk models across different subgroups without any tools. Therefore, we deve… ▽ More Disease risk models can identify high-risk patients and help clinicians provide more personalized care. However, risk models developed on one dataset may not generalize across diverse subpopulations of patients in different datasets and may have unexpected performance. It is challenging for clinical researchers to inspect risk models across different subgroups without any tools. Therefore, we developed an interactive visualization system called RMExplorer (Risk Model Explorer) to enable interactive risk model assessment. Specifically, the system allows users to define subgroups of patients by selecting clinical, demographic, or other characteristics, to explore the performance and fairness of risk models on the subgroups, and to understand the feature contributions to risk scores. To demonstrate the usefulness of the tool, we conduct a case study, where we use RMExplorer to explore three atrial fibrillation risk models by applying them to the UK Biobank dataset of 445,329 individuals. RMExplorer can help researchers to evaluate the performance and biases of risk models on subpopulations of interest in their data. △ Less

Submitted 13 September, 2022; originally announced September 2022.

Comments: IEEE VIS 2022 Short

DASH: Visual Analytics for Debiasing Image Classification via User-Driven Synthetic Data Augmentation

Authors: Bum Chul Kwon, Jungsoo Lee, Chaeyeon Chung, Nyoungwoo Lee, Ho-** Choi, Jaegul Choo

Abstract: Image classification models often learn to predict a class based on irrelevant co-occurrences between input features and an output class in training data. We call the unwanted correlations "data biases," and the visual features causing data biases "bias factors." It is challenging to identify and mitigate biases automatically without human intervention. Therefore, we conducted a design study to fi… ▽ More Image classification models often learn to predict a class based on irrelevant co-occurrences between input features and an output class in training data. We call the unwanted correlations "data biases," and the visual features causing data biases "bias factors." It is challenging to identify and mitigate biases automatically without human intervention. Therefore, we conducted a design study to find a human-in-the-loop solution. First, we identified user tasks that capture the bias mitigation process for image classification models with three experts. Then, to support the tasks, we developed a visual analytics system called DASH that allows users to visually identify bias factors, to iteratively generate synthetic images using a state-of-the-art image-to-image translation model, and to supervise the model training process for improving the classification accuracy. Our quantitative evaluation and qualitative study with ten participants demonstrate the usefulness of DASH and provide lessons for future work. △ Less

Submitted 13 September, 2022; originally announced September 2022.

Comments: 5 pages, 3 figures, EuroVis 2022 Short, Honorable Mention

LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Models

Authors: Gunho Park, Baeseong Park, Minsub Kim, Sungjae Lee, Jeonghoon Kim, Beomseok Kwon, Se Jung Kwon, Byeongwook Kim, Youngjoo Lee, Dongsoo Lee

Abstract: Recent advances in self-supervised learning and the Transformer architecture have significantly improved natural language processing (NLP), achieving remarkably low perplexity. However, the growing size of NLP models introduces a memory wall problem during the generation phase. To mitigate this issue, recent efforts have focused on quantizing model weights to sub-4-bit precision while preserving f… ▽ More Recent advances in self-supervised learning and the Transformer architecture have significantly improved natural language processing (NLP), achieving remarkably low perplexity. However, the growing size of NLP models introduces a memory wall problem during the generation phase. To mitigate this issue, recent efforts have focused on quantizing model weights to sub-4-bit precision while preserving full precision for activations, resulting in practical speed-ups during inference on a single GPU. However, these improvements primarily stem from reduced memory movement, which necessitates a resource-intensive dequantization process rather than actual computational reduction. In this paper, we introduce LUT-GEMM, an efficient kernel for quantized matrix multiplication, which not only eliminates the resource-intensive dequantization process but also reduces computational costs compared to previous kernels for weight-only quantization. Furthermore, we proposed group-wise quantization to offer a flexible trade-off between compression ratio and accuracy. The impact of LUT-GEMM is facilitated by implementing high compression ratios through low-bit quantization and efficient LUT-based operations. We show experimentally that when applied to the OPT-175B model with 3-bit quantization, LUT-GEMM substantially accelerates token generation latency, achieving a remarkable 2.1$\times$ improvement on a single GPU when compared to OPTQ, which relies on the costly dequantization process. △ Less

Submitted 1 April, 2024; v1 submitted 19 June, 2022; originally announced June 2022.

Comments: ICLR 2024

Factorization Approach for Sparse Spatio-Temporal Brain-Computer Interface

Authors: Byeong-Hoo Lee, Jeong-Hyun Cho, Byoung-Hee Kwon, Seong-Whan Lee

Abstract: Recently, advanced technologies have unlimited potential in solving various problems with a large amount of data. However, these technologies have yet to show competitive performance in brain-computer interfaces (BCIs) which deal with brain signals. Basically, brain signals are difficult to collect in large quantities, in particular, the amount of information would be sparse in spontaneous BCIs. I… ▽ More Recently, advanced technologies have unlimited potential in solving various problems with a large amount of data. However, these technologies have yet to show competitive performance in brain-computer interfaces (BCIs) which deal with brain signals. Basically, brain signals are difficult to collect in large quantities, in particular, the amount of information would be sparse in spontaneous BCIs. In addition, we conjecture that high spatial and temporal similarities between tasks increase the prediction difficulty. We define this problem as sparse condition. To solve this, a factorization approach is introduced to allow the model to obtain distinct representations from latent space. To this end, we propose two feature extractors: A class-common module is trained through adversarial learning acting as a generator; Class-specific module utilizes loss function generated from classification so that features are extracted with traditional methods. To minimize the latent space shared by the class-common and class-specific features, the model is trained under orthogonal constraint. As a result, EEG signals are factorized into two separate latent spaces. Evaluations were conducted on a single-arm motor imagery dataset. From the results, we demonstrated that factorizing the EEG signal allows the model to extract rich and decisive features under sparse condition. △ Less

Submitted 16 June, 2022; originally announced June 2022.

Comments: 8 pages

An Empirical Study on the Relationship Between the Number of Coordinated Views and Visual Analysis

Authors: Juyoung Oh, Chunggi Lee, Hwiyeon Kim, Kihwan Kim, Osang Kwon, Eric D. Ragan, Bum Chul Kwon, Sungahn Ko

Abstract: Coordinated Multiple views (CMVs) are a visualization technique that simultaneously presents multiple visualizations in separate but linked views. There are many studies that report the advantages (e.g., usefulness for finding hidden relationships) and disadvantages (e.g., cognitive load) of CMVs. But little empirical work exists on the impact of the number of views on visual anlaysis results and… ▽ More Coordinated Multiple views (CMVs) are a visualization technique that simultaneously presents multiple visualizations in separate but linked views. There are many studies that report the advantages (e.g., usefulness for finding hidden relationships) and disadvantages (e.g., cognitive load) of CMVs. But little empirical work exists on the impact of the number of views on visual anlaysis results and processes, which results in uncertainty in the relationship between the view number and visual anlaysis. In this work, we aim at investigating the relationship between the number of coordinated views and users analytic processes and results. To achieve the goal, we implemented a CMV tool for visual anlaysis. We also provided visualization duplication in the tool to help users easily create a desired number of visualization views on-the-fly. We conducted a between-subject study with 44 participants, where we asked participants to solve five analytic problems using the visual tool. Through quantitative and qualitative analysis, we discovered the positive correlation between the number of views and analytic results. We also found that visualization duplication encourages users to create more views and to take various analysis strategies. Based on the results, we provide implications and limitations of our study. △ Less

Submitted 20 April, 2022; originally announced April 2022.

ConceptExplainer: Interactive Explanation for Deep Neural Networks from a Concept Perspective

Authors: **bin Huang, Aditi Mishra, Bum Chul Kwon, Chris Bryan

Abstract: Traditional deep learning interpretability methods which are suitable for model users cannot explain network behaviors at the global level and are inflexible at providing fine-grained explanations. As a solution, concept-based explanations are gaining attention due to their human intuitiveness and their flexibility to describe both global and local model behaviors. Concepts are groups of similarly… ▽ More Traditional deep learning interpretability methods which are suitable for model users cannot explain network behaviors at the global level and are inflexible at providing fine-grained explanations. As a solution, concept-based explanations are gaining attention due to their human intuitiveness and their flexibility to describe both global and local model behaviors. Concepts are groups of similarly meaningful pixels that express a notion, embedded within the network's latent space and have commonly been hand-generated, but have recently been discovered by automated approaches. Unfortunately, the magnitude and diversity of discovered concepts makes it difficult to navigate and make sense of the concept space. Visual analytics can serve a valuable role in bridging these gaps by enabling structured navigation and exploration of the concept space to provide concept-based insights of model behavior to users. To this end, we design, develop, and validate ConceptExplainer, a visual analytics system that enables people to interactively probe and explore the concept space to explain model behavior at the instance/class/global level. The system was developed via iterative prototy** to address a number of design challenges that model users face in interpreting the behavior of deep learning models. Via a rigorous user study, we validate how ConceptExplainer supports these challenges. Likewise, we conduct a series of usage scenarios to demonstrate how the system supports the interactive analysis of model behavior across a variety of tasks and explanation granularities, such as identifying concepts that are important to classification, identifying bias in training data, and understanding how concepts can be shared across diverse and seemingly dissimilar classes. △ Less

Submitted 24 October, 2022; v1 submitted 4 April, 2022; originally announced April 2022.

Comments: 9 pages, 6 figures

A Factorization Approach for Motor Imagery Classification

Authors: Byeong-Hoo Lee, Jeong-Hyun Cho, Byung-Hee Kwon

Abstract: Brain-computer interface uses brain signals to communicate with external devices without actual control. Many studies have been conducted to classify motor imagery based on machine learning. However, classifying imagery data with sparse spatial characteristics, such as single-arm motor imagery, remains a challenge. In this paper, we proposed a method to factorize EEG signals into two groups to cla… ▽ More Brain-computer interface uses brain signals to communicate with external devices without actual control. Many studies have been conducted to classify motor imagery based on machine learning. However, classifying imagery data with sparse spatial characteristics, such as single-arm motor imagery, remains a challenge. In this paper, we proposed a method to factorize EEG signals into two groups to classify motor imagery even if spatial features are sparse. Based on adversarial learning, we focused on extracting common features of EEG signals which are robust to noise and extracting only signal features. In addition, class-specific features were extracted which are specialized for class classification. Finally, the proposed method classifies the classes by representing the features of the two groups as one embedding space. Through experiments, we confirmed the feasibility that extracting features into two groups is advantageous for datasets that contain sparse spatial features. △ Less

Submitted 13 December, 2021; originally announced December 2021.

Comments: 4 pages

Decoding Continual Muscle Movements Related to Complex Hand Gras** from EEG Signals

Authors: Jeong-Hyun Cho, Byoung-Hee Kwon, Byeong-Hoo Lee, Seong-Whan Lee

Abstract: Brain-computer interface (BCI) is a practical pathway to interpret users' intentions by decoding motor execution (ME) or motor imagery (MI) from electroencephalogram (EEG) signals. However, develo** a BCI system driven by ME or MI is challenging, particularly in the case of containing continual and compound muscles movements. This study analyzes three gras** actions from EEG under both ME and… ▽ More Brain-computer interface (BCI) is a practical pathway to interpret users' intentions by decoding motor execution (ME) or motor imagery (MI) from electroencephalogram (EEG) signals. However, develo** a BCI system driven by ME or MI is challenging, particularly in the case of containing continual and compound muscles movements. This study analyzes three gras** actions from EEG under both ME and MI paradigms. We also investigate the classification performance in offline and pseudo-online experiments. We propose a novel approach that uses muscle activity pattern (MAP) images for the convolutional neural network (CNN) to improve classification accuracy. We record the EEG and electromyogram (EMG) signals simultaneously and create the MAP images by decoding both signals to estimate specific hand gras**. As a result, we obtained an average classification accuracy of 63.6($\pm$6.7)% in ME and 45.8($\pm$4.4)% in MI across all fifteen subjects for four classes. Also, we performed pseudo-online experiments and obtained classification accuracies of 60.5($\pm$8.4)% in ME and 42.7($\pm$6.8)% in MI. The proposed method MAP-CNN, shows stable classification performance, even in the pseudo-online experiment. We expect that MAP-CNN could be used in various BCI applications in the future. △ Less

Submitted 15 December, 2021; originally announced December 2021.

Comments: Submitted to 2022 10th IEEE International Winter Conference on Brain-Computer Interface

Decoding Visual Imagery from EEG Signals using Visual Perception Guided Network Training Method

Authors: Byoung-Hee Kwon, Jeong-Hyun Cho, Byeong-Hoo Lee

Abstract: An electroencephalogram is an effective approach that provides a bidirectional pathway between user and computer in a non-invasive way. In this study, we adopted the visual perception data for training the visual imagery decoding network. We proposed a visual perception-guided network training approach for decoding visual imagery. Visual perception decreases the power of the alpha frequency range… ▽ More An electroencephalogram is an effective approach that provides a bidirectional pathway between user and computer in a non-invasive way. In this study, we adopted the visual perception data for training the visual imagery decoding network. We proposed a visual perception-guided network training approach for decoding visual imagery. Visual perception decreases the power of the alpha frequency range of the visual cortex over time when the user performed the task, and visual imagery increases the power of the alpha frequency range of the visual cortex over time as the user performed with the task. Generated brain signals when the user performing visual imagery and visual perception have opposite brain activity tendencies, and we used these characteristics to design the proposed network. When using the proposed method, the average classification performance of visual imagery with the visual perception data was 0.7008. Our results provide the possibility of using the visual perception data as a guide of the visual imagery classification network training. △ Less

Submitted 13 December, 2021; originally announced December 2021.

Comments: 4 pages, 2 figures, 3 tables

Accelerating Optimal Experimental Design for Robust Synchronization of Uncertain Kuramoto Oscillator Model Using Machine Learning

Authors: Hyun-Myung Woo, Youngjoon Hong, Bongsuk Kwon, Byung-Jun Yoon

Abstract: Recent advances in objective-based uncertainty quantification (objective-UQ) have shown that such a goal-driven approach for quantifying model uncertainty is extremely useful in real-world problems that aim at achieving specific objectives based on complex uncertain systems. Central to this objective-UQ is the concept of mean objective cost of uncertainty (MOCU), which provides effective means of… ▽ More Recent advances in objective-based uncertainty quantification (objective-UQ) have shown that such a goal-driven approach for quantifying model uncertainty is extremely useful in real-world problems that aim at achieving specific objectives based on complex uncertain systems. Central to this objective-UQ is the concept of mean objective cost of uncertainty (MOCU), which provides effective means of quantifying the impact of uncertainty on the operational goals at hand. MOCU is especially useful for optimal experimental design (OED) as the potential efficacy of an experimental (or data acquisition) campaign can be quantified by estimating the MOCU that is expected to remain after the campaign. However, MOCU-based OED tends to be computationally expensive, which limits its practical applicability. In this paper, we propose a novel machine learning (ML) scheme that can significantly accelerate MOCU computation and expedite MOCU-based experimental design. The main idea is to use an ML model to efficiently search for the optimal robust operator under model uncertainty, a necessary step for computing MOCU. We apply the proposed ML-based OED acceleration scheme to design experiments aimed at optimally enhancing the control performance of uncertain Kuramoto oscillator models. Our results show that the proposed scheme results in up to 154-fold speed improvement without any degradation of the OED performance. △ Less

Submitted 24 October, 2021; v1 submitted 1 June, 2021; originally announced June 2021.

Report number: T-SP-28061-2021

Visual Motion Imagery Classification with Deep Neural Network based on Functional Connectivity

Authors: Byoung-Hee Kwon, Ji-Hoon Jeong, Seong-Whan Lee

Abstract: Brain-computer interfaces (BCIs) use brain signals such as electroencephalography to reflect user intention and enable two-way communication between computers and users. BCI technology has recently received much attention in healthcare applications, such as neurorehabilitation and diagnosis. BCI applications can also control external devices using only brain activity, which can help people with ph… ▽ More Brain-computer interfaces (BCIs) use brain signals such as electroencephalography to reflect user intention and enable two-way communication between computers and users. BCI technology has recently received much attention in healthcare applications, such as neurorehabilitation and diagnosis. BCI applications can also control external devices using only brain activity, which can help people with physical or mental disabilities, especially those suffering from neurological and neuromuscular diseases such as stroke and amyotrophic lateral sclerosis. Motor imagery (MI) has been widely used for BCI-based device control, but we adopted intuitive visual motion imagery to overcome the weakness of MI. In this study, we developed a three-dimensional (3D) BCI training platform to induce users to imagine upper-limb movements used in real-life activities (picking up a cell phone, pouring water, opening a door, and eating food). We collected intuitive visual motion imagery data and proposed a deep learning network based on functional connectivity as a mind-reading technique. As a result, the proposed network recorded a high classification performance on average (71.05%). Furthermore, we applied the leave-one-subject-out approach to confirm the possibility of improvements in subject-independent classification performance. This study will contribute to the development of BCI-based healthcare applications for rehabilitation, such as robotic arms and wheelchairs, or assist daily life. △ Less

Submitted 30 January, 2024; v1 submitted 4 March, 2021; originally announced March 2021.

Comments: Research has advanced further

Formation of Singularities in Plasma Ion Dynamics

Authors: Junsik Bae, Junho Choi, Bongsuk Kwon

Abstract: We study the formation of singularity for the Euler-Poisson system equipped with the Boltzmann relation, which describes the dynamics of ions in an electrostatic plasma. In general, it is known that smooth solutions to nonlinear hyperbolic equations fail to exist globally in time. We establish criteria for $C^1$ blow-up of the Euler-Poisson system, both for the isothermal and pressureless cases. I… ▽ More We study the formation of singularity for the Euler-Poisson system equipped with the Boltzmann relation, which describes the dynamics of ions in an electrostatic plasma. In general, it is known that smooth solutions to nonlinear hyperbolic equations fail to exist globally in time. We establish criteria for $C^1$ blow-up of the Euler-Poisson system, both for the isothermal and pressureless cases. In particular, our blow-up condition for the presureless model does not require that the gradient of velocity is negatively large. In fact, our result particularly implies that the smooth solutions can break down even if the gradient of initial velocity is trivial. For the isothermal case, we prove that smooth solutions leave $C^1$ class in a finite time when the gradients of the Riemann functions are initially large. △ Less

Submitted 6 June, 2022; v1 submitted 17 December, 2020; originally announced December 2020.

Comments: 25 pages, 7 figures, 2 tables, the article entitled "formation of singularities in cold ion dynamics" (14 pages, 4 figures, 1 table) reorganized including the isothermal pressure case, introduction and numerical experiments sections rewritten accordingly

Linear Stability of solitary waves for the isothermal Euler-Poisson system

Authors: Junsik Bae, Bongsuk Kwon

Abstract: We study the asymptotic linear stability of a two-parameter family of solitary waves for the isothermal Euler-Poisson system. When the linearized equations about the solitary waves are considered, the associated eigenvalue problem in $L^2$ space has a zero eigenvalue embedded in the neutral spectrum, i.e., there is no spectral gap. To resolve this issue, use is made of an exponentially weighted… ▽ More We study the asymptotic linear stability of a two-parameter family of solitary waves for the isothermal Euler-Poisson system. When the linearized equations about the solitary waves are considered, the associated eigenvalue problem in $L^2$ space has a zero eigenvalue embedded in the neutral spectrum, i.e., there is no spectral gap. To resolve this issue, use is made of an exponentially weighted $L^2$ norm so that the essential spectrum is strictly shifted into the left-half plane, and this is closely related to the fact that solitary waves exist in the super-ion-sonic regime. Furthermore, in a certain long-wavelength scaling, we show that the Evans function for the Euler-Poisson system converges to that for the Korteweg-de Vries (KdV) equation as an amplitude parameter tends to zero, from which we deduce that the origin is the only eigenvalue on its natural domain with algebraic multiplicity two. We also show that the solitary waves are spectrally stable in $L^2$ space. Moreover, we discuss (in)stability of large amplitude solitary waves. △ Less

Submitted 14 December, 2020; originally announced December 2020.

Modeling Disease Progression Trajectories from Longitudinal Observational Data

Authors: Bum Chul Kwon, Peter Achenbach, Jessica L. Dunne, William Hagopian, Markus Lundgren, Kenney Ng, Riitta Veijola, Brigitte I. Frohnert, Vibha Anand, the T1DI Study Group

Abstract: Analyzing disease progression patterns can provide useful insights into the disease processes of many chronic conditions. These analyses may help inform recruitment for prevention trials or the development and personalization of treatments for those affected. We learn disease progression patterns using Hidden Markov Models (HMM) and distill them into distinct trajectories using visualization metho… ▽ More Analyzing disease progression patterns can provide useful insights into the disease processes of many chronic conditions. These analyses may help inform recruitment for prevention trials or the development and personalization of treatments for those affected. We learn disease progression patterns using Hidden Markov Models (HMM) and distill them into distinct trajectories using visualization methods. We apply it to the domain of Type 1 Diabetes (T1D) using large longitudinal observational data from the T1DI study group. Our method discovers distinct disease progression trajectories that corroborate with recently published findings. In this paper, we describe the iterative process of develo** the model. These methods may also be applied to other chronic conditions that evolve over time. △ Less

Submitted 9 December, 2020; originally announced December 2020.

Comments: 9 pages, 5 figures, to be published in proceedings of AMIA Annual Symposium 2020

Speech Imagery Classification using Length-Wise Training based on Deep Learning

Authors: Byeong-Hoo Lee, Byeong-Hee Kwon, Do-Yeun Lee, Ji-Hoon Jeong

Abstract: Brain-computer interface uses brain signals to control external devices without actual control behavior. Recently, speech imagery has been studied for direct communication using language. Speech imagery uses brain signals generated when the user imagines speech. Unlike motor imagery, speech imagery still has unknown characteristics. Additionally, electroencephalography has intricate and non-statio… ▽ More Brain-computer interface uses brain signals to control external devices without actual control behavior. Recently, speech imagery has been studied for direct communication using language. Speech imagery uses brain signals generated when the user imagines speech. Unlike motor imagery, speech imagery still has unknown characteristics. Additionally, electroencephalography has intricate and non-stationary properties resulting in insufficient decoding performance. In addition, speech imagery is difficult to utilize spatial features. In this study, we designed length-wise training that allows the model to classify a series of a small number of words. In addition, we proposed hierarchical convolutional neural network structure and loss function to maximize the training strategy. The proposed method showed competitive performance in speech imagery classification. Hence, we demonstrated that the length of the word is a clue at improving classification performance. △ Less

Submitted 7 December, 2020; originally announced December 2020.

Comments: Submitted IEEE The 9th International Winter Conference on Brain-Computer Interface

Motor Imagery Classification Emphasizing Corresponding Frequency Domain Method based on Deep Learning Framework

Authors: Byoung-Hee Kwon, Byeong-Hoo Lee, Ji-Hoon Jeong

Abstract: The electroencephalogram, a type of non-invasive-based brain signal that has a user intention-related feature provides an efficient bidirectional pathway between user and computer. In this work, we proposed a deep learning framework based on corresponding frequency empahsize method to decode the motor imagery (MI) data from 2020 International BCI competition dataset. The MI dataset consists of 3-c… ▽ More The electroencephalogram, a type of non-invasive-based brain signal that has a user intention-related feature provides an efficient bidirectional pathway between user and computer. In this work, we proposed a deep learning framework based on corresponding frequency empahsize method to decode the motor imagery (MI) data from 2020 International BCI competition dataset. The MI dataset consists of 3-class, namely 'Cylindrical', 'Spherical', and 'Lumbrical'. We utilized power spectral density as an emphasize method and a convolutional neural network to classify the modified MI data. The results showed that MI-related frequency range was activated during MI task, and provide neurophysiological evidence to design the proposed method. When using the proposed method, the average classification performance in intra-session condition was 69.68% and the average classification performance in inter-session condition was 52.76%. Our results provided the possibility of develo** a BCI-based device control system for practical applications. △ Less

Submitted 7 December, 2020; originally announced December 2020.

Comments: Submitted IEEE The 9th International Winter Conference on Brain-Computer Interface

User-driven Analysis of Longitudinal Health Data with Hidden Markov Models for Clinical Insights

Authors: Bum Chul Kwon

Abstract: A goal of clinical researchers is to understand the progression of a disease through a set of biomarkers. Researchers often conduct observational studies, where they collect numerous samples from selected subjects throughout multiple years. Hidden Markov Models (HMMs) can be applied to discover latent states and their transition probabilities over time. However, it is challenging for clinical rese… ▽ More A goal of clinical researchers is to understand the progression of a disease through a set of biomarkers. Researchers often conduct observational studies, where they collect numerous samples from selected subjects throughout multiple years. Hidden Markov Models (HMMs) can be applied to discover latent states and their transition probabilities over time. However, it is challenging for clinical researchers to interpret the outcomes and to gain insights about the disease. Thus, this demo introduces an interactive visualization system called DPVis, which was designed to help researchers to interactively explore HMM outcomes. The demo provides guidelines of how to implement the clinician-in-the-loop approach for analyzing longitudinal, observational health data with visual analytics. △ Less

Submitted 24 July, 2020; originally announced July 2020.

Comments: Demo presented in KDD 2020 Workshop on Applied Data Science for Healthcare

Optimal Experimental Design for Uncertain Systems Based on Coupled Differential Equations

Authors: Youngjoon Hong, Bongsuk Kwon, Byung-Jun Yoon

Abstract: We consider the optimal experimental design problem for an uncertain Kuramoto model, which consists of N interacting oscillators described by coupled ordinary differential equations. The objective is to design experiments that can effectively reduce the uncertainty present in the coupling strengths between the oscillators, thereby minimizing the cost of robust control of the uncertain Kuramoto mod… ▽ More We consider the optimal experimental design problem for an uncertain Kuramoto model, which consists of N interacting oscillators described by coupled ordinary differential equations. The objective is to design experiments that can effectively reduce the uncertainty present in the coupling strengths between the oscillators, thereby minimizing the cost of robust control of the uncertain Kuramoto model. We demonstrate the importance of quantifying the operational impact of the potential experiments in designing optimal experiments. △ Less

Submitted 27 March, 2021; v1 submitted 12 July, 2020; originally announced July 2020.

Decoding of Intuitive Visual Motion Imagery Using Convolutional Neural Network under 3D-BCI Training Environment

Authors: Byoung-Hee Kwon, Ji-Hoon Jeong, Jeong-Hyun Cho, Seong-Whan Lee

Abstract: In this study, we adopted visual motion imagery, which is a more intuitive brain-computer interface (BCI) paradigm, for decoding the intuitive user intention. We developed a 3-dimensional BCI training platform and applied it to assist the user in performing more intuitive imagination in the visual motion imagery experiment. The experimental tasks were selected based on the movements that we common… ▽ More In this study, we adopted visual motion imagery, which is a more intuitive brain-computer interface (BCI) paradigm, for decoding the intuitive user intention. We developed a 3-dimensional BCI training platform and applied it to assist the user in performing more intuitive imagination in the visual motion imagery experiment. The experimental tasks were selected based on the movements that we commonly used in daily life, such as picking up a phone, opening a door, eating food, and pouring water. Nine subjects participated in our experiment. We presented statistical evidence that visual motion imagery has a high correlation from the prefrontal and occipital lobes. In addition, we selected the most appropriate electroencephalography channels using a functional connectivity approach for visual motion imagery decoding and proposed a convolutional neural network architecture for classification. As a result, the averaged classification performance of the proposed architecture for 4 classes from 16 channels was 67.50 % across all subjects. This result is encouraging, and it shows the possibility of develo** a BCI-based device control system for practical applications such as neuroprosthesis and a robotic arm. △ Less

Submitted 15 May, 2020; originally announced May 2020.

Comments: Submitted to IEEE International Conference on System, Man, and Cybernetics (IEEE SMC 2020)

A Novel Framework for Visual Motion Imagery Classification Using 3D Virtual BCI Platform

Authors: Byoung-Hee Kwon, Ji-Hoon Jeong, Dong-Joo Kim

Abstract: In this study, 3D brain-computer interface (BCI) training platforms were used to stimulate the subjects for visual motion imagery and visual perception. We measured the activation brain region and alpha-band power activity when the subjects perceived and imagined the stimuli. Based on this, 4-class were classified in visual stimuli session and visual motion imagery session respectively. The result… ▽ More In this study, 3D brain-computer interface (BCI) training platforms were used to stimulate the subjects for visual motion imagery and visual perception. We measured the activation brain region and alpha-band power activity when the subjects perceived and imagined the stimuli. Based on this, 4-class were classified in visual stimuli session and visual motion imagery session respectively. The results showed that the occipital region is involved in visual perception and visual motion imagery, and alpha-band power is increased in visual motion imagery session and decreased in visual motion stimuli session. Compared with the performance of visual motion imagery and motor imagery, visual motion imagery has higher performance than motor imagery. The binary class was classified using one versus rest approach as well as analysis of brain activation to prove that visual-related brain wave signals are meaningful, and the results were significant. △ Less

Submitted 3 February, 2020; originally announced February 2020.

Comments: 5 pages, 4 figures, 2 tables

GUIComp: A GUI Design Assistant with Real-Time, Multi-Faceted Feedback

Authors: Chunggi Lee, Sanghoon Kim, Dongyun Han, Hongjun Yang, Young-Woo Park, Bum Chul Kwon, Sungahn Ko

Abstract: Users may face challenges while designing graphical user interfaces, due to a lack of relevant experience and guidance. This paper aims to investigate the issues that users with no experience face during the design process, and how to resolve them. To this end, we conducted semi-structured interviews, based on which we built a GUI prototy** assistance tool called GUIComp. This tool can be connec… ▽ More Users may face challenges while designing graphical user interfaces, due to a lack of relevant experience and guidance. This paper aims to investigate the issues that users with no experience face during the design process, and how to resolve them. To this end, we conducted semi-structured interviews, based on which we built a GUI prototy** assistance tool called GUIComp. This tool can be connected to GUI design software as an extension, and it provides real-time, multi-faceted feedback on a user's current design. Additionally, we conducted two user studies, in which we asked participants to create mobile GUIs with or without GUIComp, and requested online workers to assess the created GUIs. The experimental results show that GUIComp facilitated iterative design and the participants with GUIComp had better a user experience and produced more acceptable designs than those who did not. △ Less

Submitted 16 January, 2020; originally announced January 2020.

Comments: 10 pages, 3 figures, accepted paper to CHI 2020 conference

Geono-Cluster: Interactive Visual Cluster Analysis for Biologists

Authors: Bahador Saket, Subhajit Das, Bum Chul Kwon, Alex Endert

Abstract: Biologists often perform clustering analysis to derive meaningful patterns, relationships, and structures from data instances and attributes. Though clustering plays a pivotal role in biologists' data exploration, it takes non-trivial efforts for biologists to find the best grou** in their data using existing tools. Visual cluster analysis is currently performed either programmatically or throug… ▽ More Biologists often perform clustering analysis to derive meaningful patterns, relationships, and structures from data instances and attributes. Though clustering plays a pivotal role in biologists' data exploration, it takes non-trivial efforts for biologists to find the best grou** in their data using existing tools. Visual cluster analysis is currently performed either programmatically or through menus and dialogues in many tools, which require parameter adjustments over several steps of trial-and-error. In this paper, we introduce Geono-Cluster, a novel visual analysis tool designed to support cluster analysis for biologists who do not have formal data science training. Geono-Cluster enables biologists to apply their domain expertise into clustering results by visually demonstrating how their expected clustering outputs should look like with a small sample of data instances. The system then predicts users' intentions and generates potential clustering results. Our study follows the design study protocol to derive biologists' tasks and requirements, design the system, and evaluate the system with experts on their own dataset. Results of our study with six biologists provide initial evidence that Geono-Cluster enables biologists to create, refine, and evaluate clustering results to effectively analyze their data and gain data-driven insights. At the end, we discuss lessons learned and the implications of our study. △ Less

Submitted 3 November, 2019; originally announced November 2019.

SANVis: Visual Analytics for Understanding Self-Attention Networks

Authors: Cheonbok Park, Inyoup Na, Yongjang Jo, Sungbok Shin, Jaehyo Yoo, Bum Chul Kwon, Jian Zhao, Hyungjong Noh, Yeonsoo Lee, Jaegul Choo

Abstract: Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications. Recently, they have been further evolved into an advanced approach called multi-head self-attention networks, which can encode a set of input vectors, e.g., word vectors in a sentence, into another set o… ▽ More Attention networks, a deep neural network architecture inspired by humans' attention mechanism, have seen significant success in image captioning, machine translation, and many other applications. Recently, they have been further evolved into an advanced approach called multi-head self-attention networks, which can encode a set of input vectors, e.g., word vectors in a sentence, into another set of vectors. Such encoding aims at simultaneously capturing diverse syntactic and semantic features within a set, each of which corresponds to a particular attention head, forming altogether multi-head attention. Meanwhile, the increased model complexity prevents users from easily understanding and manipulating the inner workings of models. To tackle the challenges, we present a visual analytics system called SANVis, which helps users understand the behaviors and the characteristics of multi-head self-attention networks. Using a state-of-the-art self-attention model called Transformer, we demonstrate usage scenarios of SANVis in machine translation tasks. Our system is available at http://short.sanvis.org △ Less

Submitted 13 September, 2019; originally announced September 2019.

Comments: VAST Short - IEEE VIS 2019

Thumbnails for Data Stories: A Survey of Current Practices

Authors: Hwiyeon Kim, Juyoung Oh, Yunha Han, Sungahn Ko, Matthew Brehmer, Bum Chul Kwon

Abstract: When people browse online news, small thumbnail images accompanying links to articles attract their attention and help them to decide which articles to read. As an increasing proportion of online news can be construed as data journalism, we have witnessed a corresponding increase in the incorporation of visualization in article thumbnails. However, there is little research to support alternative d… ▽ More When people browse online news, small thumbnail images accompanying links to articles attract their attention and help them to decide which articles to read. As an increasing proportion of online news can be construed as data journalism, we have witnessed a corresponding increase in the incorporation of visualization in article thumbnails. However, there is little research to support alternative design choices for visualization thumbnails, which include resizing, crop**, simplifying, and embellishing charts appearing within the body of the associated article. We therefore sought to better understand these design choices and determine what makes a visualization thumbnail inviting and interpretable. This paper presents our findings from a survey of visualization thumbnails collected online and from conversations with data journalists and news graphics designers. Our study reveals that there exists an uncharted design space, one that is in need of further empirical study. Our work can thus be seen as a first step toward providing structured guidance on how to design thumbnails for data stories. △ Less

Submitted 19 August, 2019; originally announced August 2019.

Comments: To appear at IEEE VIS 2019

DPVis: Visual Analytics with Hidden Markov Models for Disease Progression Pathways

Authors: Bum Chul Kwon, Vibha Anand, Kristen A Severson, Soumya Ghosh, Zhaonan Sun, Brigitte I Frohnert, Markus Lundgren, Kenney Ng

Abstract: Clinical researchers use disease progression models to understand patient status and characterize progression patterns from longitudinal health records. One approach for disease progression modeling is to describe patient status using a small number of states that represent distinctive distributions over a set of observed measures. Hidden Markov models (HMMs) and its variants are a class of models… ▽ More Clinical researchers use disease progression models to understand patient status and characterize progression patterns from longitudinal health records. One approach for disease progression modeling is to describe patient status using a small number of states that represent distinctive distributions over a set of observed measures. Hidden Markov models (HMMs) and its variants are a class of models that both discover these states and make inferences of health states for patients. Despite the advantages of using the algorithms for discovering interesting patterns, it still remains challenging for medical experts to interpret model outputs, understand complex modeling parameters, and clinically make sense of the patterns. To tackle these problems, we conducted a design study with clinical scientists, statisticians, and visualization experts, with the goal to investigate disease progression pathways of chronic diseases, namely type 1 diabetes (T1D), Huntington's disease, Parkinson's disease, and chronic obstructive pulmonary disease (COPD). As a result, we introduce DPVis which seamlessly integrates model parameters and outcomes of HMMs into interpretable and interactive visualizations. In this study, we demonstrate that DPVis is successful in evaluating disease progression models, visually summarizing disease states, interactively exploring disease progression patterns, and building, analyzing, and comparing clinically relevant patient subgroups. △ Less

Submitted 9 April, 2020; v1 submitted 25 April, 2019; originally announced April 2019.

Comments: to appear at IEEE Transactions on Visualization and Computer Graphics