-
Burst Image Super-Resolution with Base Frame Selection
Authors:
Sanghyun Kim,
Min Jung Lee,
Woohyeok Kim,
Deunsol Jung,
Jaesung Rim,
Sunghyun Cho,
Minsu Cho
Abstract:
Burst image super-resolution has been a topic of active research in recent years due to its ability to obtain a high-resolution image by using complementary information between multiple frames in the burst. In this work, we explore using burst shots with non-uniform exposures to confront real-world practical scenarios by introducing a new benchmark dataset, dubbed Non-uniformly Exposed Burst Image…
▽ More
Burst image super-resolution has been a topic of active research in recent years due to its ability to obtain a high-resolution image by using complementary information between multiple frames in the burst. In this work, we explore using burst shots with non-uniform exposures to confront real-world practical scenarios by introducing a new benchmark dataset, dubbed Non-uniformly Exposed Burst Image (NEBI), that includes the burst frames at varying exposure times to obtain a broader range of irradiance and motion characteristics within a scene. As burst shots with non-uniform exposures exhibit varying levels of degradation, fusing information of the burst shots into the first frame as a base frame may not result in optimal image quality. To address this limitation, we propose a Frame Selection Network (FSN) for non-uniform scenarios. This network seamlessly integrates into existing super-resolution methods in a plug-and-play manner with low computational costs. The comparative analysis reveals the effectiveness of the nonuniform setting for the practical scenario and our FSN on synthetic-/real- NEBI datasets.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
Improving Demand Forecasting in Open Systems with Cartogram-Enhanced Deep Learning
Authors:
Sangjoon Park,
Yongsung Kwon,
Hyungjoon Soh,
Mi ** Lee,
Seung-Woo Son
Abstract:
Predicting temporal patterns across various domains poses significant challenges due to their nuanced and often nonlinear trajectories. To address this challenge, prediction frameworks have been continuously refined, employing data-driven statistical methods, mathematical models, and machine learning. Recently, as one of the challenging systems, shared transport systems such as public bicycles hav…
▽ More
Predicting temporal patterns across various domains poses significant challenges due to their nuanced and often nonlinear trajectories. To address this challenge, prediction frameworks have been continuously refined, employing data-driven statistical methods, mathematical models, and machine learning. Recently, as one of the challenging systems, shared transport systems such as public bicycles have gained prominence due to urban constraints and environmental concerns. Predicting rental and return patterns at bicycle stations remains a formidable task due to the system's openness and imbalanced usage patterns across stations. In this study, we propose a deep learning framework to predict rental and return patterns by leveraging cartogram approaches. The cartogram approach facilitates the prediction of demand for newly installed stations with no training data as well as long-period prediction, which has not been achieved before. We apply this method to public bicycle rental-and-return data in Seoul, South Korea, employing a spatial-temporal convolutional graph attention network. Our improved architecture incorporates batch attention and modified node feature updates for better prediction accuracy across different time scales. We demonstrate the effectiveness of our framework in predicting temporal patterns and its potential applications.
△ Less
Submitted 26 May, 2024; v1 submitted 24 March, 2024;
originally announced March 2024.
-
Robustified ANNs Reveal Wormholes Between Human Category Percepts
Authors:
Guy Gaziv,
Michael J. Lee,
James J. DiCarlo
Abstract:
The visual object category reports of artificial neural networks (ANNs) are notoriously sensitive to tiny, adversarial image perturbations. Because human category reports (aka human percepts) are thought to be insensitive to those same small-norm perturbations -- and locally stable in general -- this argues that ANNs are incomplete scientific models of human visual perception. Consistent with this…
▽ More
The visual object category reports of artificial neural networks (ANNs) are notoriously sensitive to tiny, adversarial image perturbations. Because human category reports (aka human percepts) are thought to be insensitive to those same small-norm perturbations -- and locally stable in general -- this argues that ANNs are incomplete scientific models of human visual perception. Consistent with this, we show that when small-norm image perturbations are generated by standard ANN models, human object category percepts are indeed highly stable. However, in this very same "human-presumed-stable" regime, we find that robustified ANNs reliably discover low-norm image perturbations that strongly disrupt human percepts. These previously undetectable human perceptual disruptions are massive in amplitude, approaching the same level of sensitivity seen in robustified ANNs. Further, we show that robustified ANNs support precise perceptual state interventions: they guide the construction of low-norm image perturbations that strongly alter human category percepts toward specific prescribed percepts. These observations suggest that for arbitrary starting points in image space, there exists a set of nearby "wormholes", each leading the subject from their current category perceptual state into a semantically very different state. Moreover, contemporary ANN models of biological visual processing are now accurate enough to consistently guide us to those portals.
△ Less
Submitted 4 October, 2023; v1 submitted 13 August, 2023;
originally announced August 2023.
-
Rediscovery of CNN's Versatility for Text-based Encoding of Raw Electronic Health Records
Authors:
Eunbyeol Cho,
Min Jae Lee,
Kyunghoon Hur,
Jiyoun Kim,
**sung Yoon,
Edward Choi
Abstract:
Making the most use of abundant information in electronic health records (EHR) is rapidly becoming an important topic in the medical domain. Recent work presented a promising framework that embeds entire features in raw EHR data regardless of its form and medical code standards. The framework, however, only focuses on encoding EHR with minimal preprocessing and fails to consider how to learn effic…
▽ More
Making the most use of abundant information in electronic health records (EHR) is rapidly becoming an important topic in the medical domain. Recent work presented a promising framework that embeds entire features in raw EHR data regardless of its form and medical code standards. The framework, however, only focuses on encoding EHR with minimal preprocessing and fails to consider how to learn efficient EHR representation in terms of computation and memory usage. In this paper, we search for a versatile encoder not only reducing the large data into a manageable size but also well preserving the core information of patients to perform diverse clinical tasks. We found that hierarchically structured Convolutional Neural Network (CNN) often outperforms the state-of-the-art model on diverse tasks such as reconstruction, prediction, and generation, even with fewer parameters and less training time. Moreover, it turns out that making use of the inherent hierarchy of EHR data can boost the performance of any kind of backbone models and clinical tasks performed. Through extensive experiments, we present concrete evidence to generalize our research findings into real-world practice. We give a clear guideline on building the encoder based on the research findings captured while exploring numerous settings.
△ Less
Submitted 10 May, 2023; v1 submitted 14 March, 2023;
originally announced March 2023.
-
UniHPF : Universal Healthcare Predictive Framework with Zero Domain Knowledge
Authors:
Kyunghoon Hur,
Jungwoo Oh,
Junu Kim,
Jiyoun Kim,
Min Jae Lee,
Eunbyeol Cho,
Seong-Eun Moon,
Young-Hak Kim,
Edward Choi
Abstract:
Despite the abundance of Electronic Healthcare Records (EHR), its heterogeneity restricts the utilization of medical data in building predictive models. To address this challenge, we propose Universal Healthcare Predictive Framework (UniHPF), which requires no medical domain knowledge and minimal pre-processing for multiple prediction tasks. Experimental results demonstrate that UniHPF is capable…
▽ More
Despite the abundance of Electronic Healthcare Records (EHR), its heterogeneity restricts the utilization of medical data in building predictive models. To address this challenge, we propose Universal Healthcare Predictive Framework (UniHPF), which requires no medical domain knowledge and minimal pre-processing for multiple prediction tasks. Experimental results demonstrate that UniHPF is capable of building large-scale EHR models that can process any form of medical data from distinct EHR systems. We believe that our findings can provide helpful insights for further research on the multi-source learning of EHRs.
△ Less
Submitted 15 November, 2022;
originally announced November 2022.
-
Tangential Wasserstein Projections
Authors:
Florian Gunsilius,
Meng Hsuan Hsieh,
Myung ** Lee
Abstract:
We develop a notion of projections between sets of probability measures using the geometric properties of the 2-Wasserstein space. It is designed for general multivariate probability measures, is computationally efficient to implement, and provides a unique solution in regular settings. The idea is to work on regular tangent cones of the Wasserstein space using generalized geodesics. Its structure…
▽ More
We develop a notion of projections between sets of probability measures using the geometric properties of the 2-Wasserstein space. It is designed for general multivariate probability measures, is computationally efficient to implement, and provides a unique solution in regular settings. The idea is to work on regular tangent cones of the Wasserstein space using generalized geodesics. Its structure and computational properties make the method applicable in a variety of settings, from causal inference to the analysis of object data. An application to estimating causal effects yields a generalization of the notion of synthetic controls to multivariate data with individual-level heterogeneity, as well as a way to estimate optimal weights jointly over all time periods.
△ Less
Submitted 2 August, 2022; v1 submitted 29 July, 2022;
originally announced July 2022.
-
GenHPF: General Healthcare Predictive Framework with Multi-task Multi-source Learning
Authors:
Kyunghoon Hur,
Jungwoo Oh,
Junu Kim,
Jiyoun Kim,
Min Jae Lee,
Eunbyeol Cho,
Seong-Eun Moon,
Young-Hak Kim,
Louis Atallah,
Edward Choi
Abstract:
Despite the remarkable progress in the development of predictive models for healthcare, applying these algorithms on a large scale has been challenging. Algorithms trained on a particular task, based on specific data formats available in a set of medical records, tend to not generalize well to other tasks or databases in which the data fields may differ. To address this challenge, we propose Gener…
▽ More
Despite the remarkable progress in the development of predictive models for healthcare, applying these algorithms on a large scale has been challenging. Algorithms trained on a particular task, based on specific data formats available in a set of medical records, tend to not generalize well to other tasks or databases in which the data fields may differ. To address this challenge, we propose General Healthcare Predictive Framework (GenHPF), which is applicable to any EHR with minimal preprocessing for multiple prediction tasks. GenHPF resolves heterogeneity in medical codes and schemas by converting EHRs into a hierarchical textual representation while incorporating as many features as possible. To evaluate the efficacy of GenHPF, we conduct multi-task learning experiments with single-source and multi-source settings, on three publicly available EHR datasets with different schemas for 12 clinically meaningful prediction tasks. Our framework significantly outperforms baseline models that utilize domain knowledge in multi-source learning, improving average AUROC by 1.2%P in pooled learning and 2.6%P in transfer learning while also showing comparable results when trained on a single EHR dataset. Furthermore, we demonstrate that self-supervised pretraining using multi-source datasets is effective when combined with GenHPF, resulting in a 0.6%P AUROC improvement compared to models without pretraining. By eliminating the need for preprocessing and feature engineering, we believe that this work offers a solid framework for multi-task and multi-source learning that can be leveraged to speed up the scaling and usage of predictive algorithms in healthcare.
△ Less
Submitted 15 November, 2023; v1 submitted 20 July, 2022;
originally announced July 2022.
-
Adversarially trained neural representations may already be as robust as corresponding biological neural representations
Authors:
Chong Guo,
Michael J. Lee,
Guillaume Leclerc,
Joel Dapello,
Yug Rao,
Aleksander Madry,
James J. DiCarlo
Abstract:
Visual systems of primates are the gold standard of robust perception. There is thus a general belief that mimicking the neural representations that underlie those systems will yield artificial visual systems that are adversarially robust. In this work, we develop a method for performing adversarial visual attacks directly on primate brain activity. We then leverage this method to demonstrate that…
▽ More
Visual systems of primates are the gold standard of robust perception. There is thus a general belief that mimicking the neural representations that underlie those systems will yield artificial visual systems that are adversarially robust. In this work, we develop a method for performing adversarial visual attacks directly on primate brain activity. We then leverage this method to demonstrate that the above-mentioned belief might not be well founded. Specifically, we report that the biological neurons that make up visual systems of primates exhibit susceptibility to adversarial perturbations that is comparable in magnitude to existing (robustly trained) artificial neural networks.
△ Less
Submitted 19 June, 2022;
originally announced June 2022.
-
Semantic 3D Map Change Detection and Update based on Smartphone Visual Positioning System
Authors:
Max Jwo Lem Lee,
Li-Ta Hsu
Abstract:
Accurate localization and 3D maps are increasingly needed for various artificial intelligence based IoT applications such as augmented reality, intelligent transportation, crowd monitoring, robotics, etc. This article proposes a novel semantic 3D map change detection and update based on a smartphone visual positioning system (VPS) for the outdoor and indoor environments. The proposed method presen…
▽ More
Accurate localization and 3D maps are increasingly needed for various artificial intelligence based IoT applications such as augmented reality, intelligent transportation, crowd monitoring, robotics, etc. This article proposes a novel semantic 3D map change detection and update based on a smartphone visual positioning system (VPS) for the outdoor and indoor environments. The proposed method presents an alternate solution to SLAM for map update in terms of efficiency, cost, availability, and map reuse. Building on existing 3D maps of recent years, a system is designed to use artificial intelligence to identify high-level semantics in images for positioning and map change detection. Then, a virtual LIDAR that estimates the depth of objects in the 3D map is used to generate a compact point cloud to update changes in the scene. We present an excellent performance of localization with respect to other state-of-the-art smartphone positioning solutions to accurately update semantic 3D maps. It is shown that the proposed solution can position users within 1.9m, and update objects with an average error of 2.1m.
△ Less
Submitted 21 March, 2021;
originally announced March 2021.
-
Semantic-Based VPS for Smartphone Localization in Challenging Urban Environments
Authors:
Max Jwo Lem Lee,
Li-Ta Hsu,
Hoi-Fung Ng,
Shang Lee
Abstract:
Accurate smartphone-based outdoor localization system in deep urban canyons are increasingly needed for various IoT applications such as augmented reality, intelligent transportation, etc. The recently developed feature-based visual positioning system (VPS) by Google detects edges from smartphone images to match with pre-surveyed edges in their map database. As smart cities develop, the building i…
▽ More
Accurate smartphone-based outdoor localization system in deep urban canyons are increasingly needed for various IoT applications such as augmented reality, intelligent transportation, etc. The recently developed feature-based visual positioning system (VPS) by Google detects edges from smartphone images to match with pre-surveyed edges in their map database. As smart cities develop, the building information modeling (BIM) becomes widely available, which provides an opportunity for a new semantic-based VPS. This article proposes a novel 3D city model and semantic-based VPS for accurate and robust pose estimation in urban canyons where global navigation satellite system (GNSS) tends to fail. In the offline stage, a material segmented city model is used to generate segmented images. In the online stage, an image is taken with a smartphone camera that provides textual information about the surrounding environment. The approach utilizes computer vision algorithms to rectify and hand segment between the different types of material identified in the smartphone image. A semantic-based VPS method is then proposed to match the segmented generated images with the segmented smartphone image. Each generated image holds a pose that contains the latitude, longitude, altitude, yaw, pitch, and roll. The candidate with the maximum likelihood is regarded as the precise pose of the user. The positioning results achieves 2.0m level accuracy in common high rise along street, 5.5m in foliage dense environment and 15.7m in alleyway. A 45% positioning improvement to current state-of-the-art method. The estimation of yaw achieves 2.3° level accuracy, 8 times the improvement to smartphone IMU.
△ Less
Submitted 21 November, 2020;
originally announced November 2020.
-
Uncovering hidden dependency in weighted networks via information entropy
Authors:
Mi ** Lee,
Eun Lee,
Byunghwee Lee,
Hawoong Jeong,
Deok-Sun Lee,
Sang Hoon Lee
Abstract:
Interactions between elements, which are usually represented by networks, have to delineate potentially unequal relationships in terms of their relative importance or direction. The intrinsic unequal relationships of such kind, however, are opaque or hidden in numerous real systems. For instance, when a node in a network with limited interaction capacity spends its capacity to its neighboring node…
▽ More
Interactions between elements, which are usually represented by networks, have to delineate potentially unequal relationships in terms of their relative importance or direction. The intrinsic unequal relationships of such kind, however, are opaque or hidden in numerous real systems. For instance, when a node in a network with limited interaction capacity spends its capacity to its neighboring nodes, the allocation of the total amount of interactions to them can be vastly diverse. Even if such potentially heterogeneous interactions epitomized by weighted networks are observable, as a result of the aforementioned ego-centric allocation of interactions, the relative importance or dependency between two interacting nodes can only be implicitly accessible. In this work, we precisely pinpoint such relative dependency by proposing the framework to discover hidden dependent relations extracted from weighted networks. For a given weighted network, we provide a systematic criterion to select the most essential interactions for individual nodes based on the concept of information entropy. The criterion is symbolized by assigning the effective number of neighbors or the effective out-degree to each node, and the resultant directed subnetwork decodes the hidden dependent relations by leaving only the most essential directed interactions. We apply our methodology to two time-stamped empirical network data, namely the international trade relations between nations in the world trade web (WTW) and the network of people in the historical record of Korea, Annals of the Joseon Dynasty (AJD). Based on the data analysis, we discover that the properties of mutual dependency encoded in the two systems are vastly different.
△ Less
Submitted 29 November, 2021; v1 submitted 25 August, 2020;
originally announced August 2020.
-
Data-Driven Discovery of Molecular Photoswitches with Multioutput Gaussian Processes
Authors:
Ryan-Rhys Griffiths,
Jake L. Greenfield,
Aditya R. Thawani,
Arian R. Jamasb,
Henry B. Moss,
Anthony Bourached,
Penelope Jones,
William McCorkindale,
Alexander A. Aldrick,
Matthew J. Fuchter Alpha A. Lee
Abstract:
Photoswitchable molecules display two or more isomeric forms that may be accessed using light. Separating the electronic absorption bands of these isomers is key to selectively addressing a specific isomer and achieving high photostationary states whilst overall red-shifting the absorption bands serves to limit material damage due to UV-exposure and increases penetration depth in photopharmacologi…
▽ More
Photoswitchable molecules display two or more isomeric forms that may be accessed using light. Separating the electronic absorption bands of these isomers is key to selectively addressing a specific isomer and achieving high photostationary states whilst overall red-shifting the absorption bands serves to limit material damage due to UV-exposure and increases penetration depth in photopharmacological applications. Engineering these properties into a system through synthetic design however, remains a challenge. Here, we present a data-driven discovery pipeline for molecular photoswitches underpinned by dataset curation and multitask learning with Gaussian processes. In the prediction of electronic transition wavelengths, we demonstrate that a multioutput Gaussian process (MOGP) trained using labels from four photoswitch transition wavelengths yields the strongest predictive performance relative to single-task models as well as operationally outperforming time-dependent density functional theory (TD-DFT) in terms of the wall-clock time for prediction. We validate our proposed approach experimentally by screening a library of commercially available photoswitchable molecules. Through this screen, we identified several motifs that displayed separated electronic absorption bands of their isomers, exhibited red-shifted absorptions, and are suited for information transfer and photopharmacological applications. Our curated dataset, code, as well as all models are made available at https://github.com/Ryan-Rhys/The-Photoswitch-Dataset
△ Less
Submitted 7 August, 2022; v1 submitted 28 June, 2020;
originally announced August 2020.
-
Predicting Abandonment in Online Coding Tutorials
Authors:
An Yan,
Michael J. Lee,
Andrew J. Ko
Abstract:
Learners regularly abandon online coding tutorials when they get bored or frustrated, but there are few techniques for anticipating this abandonment to intervene. In this paper, we examine the feasibility of predicting abandonment with machine-learned classifiers. Using interaction logs from an online programming game, we extracted a collection of features that are potentially related to learner a…
▽ More
Learners regularly abandon online coding tutorials when they get bored or frustrated, but there are few techniques for anticipating this abandonment to intervene. In this paper, we examine the feasibility of predicting abandonment with machine-learned classifiers. Using interaction logs from an online programming game, we extracted a collection of features that are potentially related to learner abandonment and engagement, then developed classifiers for each level. Across the first five levels of the game, our classifiers successfully predicted 61% to 76% of learners who did not complete the next level, achieving an average AUC of 0.68. In these classifiers, features negatively associated with abandonment included account activation and help-seeking behaviors, whereas features positively associated with abandonment included features indicating difficulty and disengagement. These findings highlight the feasibility of providing timely intervention to learners likely to quit.
△ Less
Submitted 13 July, 2017;
originally announced July 2017.
-
Complementary algorithms for graphs and percolation
Authors:
Michael J. Lee
Abstract:
A pair of complementary algorithms are presented. One of the pair is a fast method for connecting graphs with an edge. The other is a fast method for removing edges from a graph. Both algorithms employ the same tree based graph representation and so, in concert, can arbitrarily modify any graph. Since the clusters of a percolation model may be described as simple connected graphs, an efficient M…
▽ More
A pair of complementary algorithms are presented. One of the pair is a fast method for connecting graphs with an edge. The other is a fast method for removing edges from a graph. Both algorithms employ the same tree based graph representation and so, in concert, can arbitrarily modify any graph. Since the clusters of a percolation model may be described as simple connected graphs, an efficient Monte Carlo scheme can be constructed that uses the algorithms to sweep the occupation probability back and forth between two turning points. This approach concentrates computational sampling time within a region of interest. A high precision value of pc = 0.59274603(9) was thus obtained, by Mersenne twister, for the two dimensional square site percolation threshold.
△ Less
Submitted 3 August, 2007;
originally announced August 2007.
