-
Doubly Perturbed Task Free Continual Learning
Authors:
Byung Hyun Lee,
Min-hwan Oh,
Se Young Chun
Abstract:
Task Free online continual learning (TF-CL) is a challenging problem where the model incrementally learns tasks without explicit task information. Although training with entire data from the past, present as well as future is considered as the gold standard, naive approaches in TF-CL with the current samples may be conflicted with learning with samples in the future, leading to catastrophic forget…
▽ More
Task Free online continual learning (TF-CL) is a challenging problem where the model incrementally learns tasks without explicit task information. Although training with entire data from the past, present as well as future is considered as the gold standard, naive approaches in TF-CL with the current samples may be conflicted with learning with samples in the future, leading to catastrophic forgetting and poor plasticity. Thus, a proactive consideration of an unseen future sample in TF-CL becomes imperative. Motivated by this intuition, we propose a novel TF-CL framework considering future samples and show that injecting adversarial perturbations on both input data and decision-making is effective. Then, we propose a novel method named Doubly Perturbed Continual Learning (DPCL) to efficiently implement these input and decision-making perturbations. Specifically, for input perturbation, we propose an approximate perturbation method that injects noise into the input data as well as the feature vector and then interpolates the two perturbed samples. For decision-making process perturbation, we devise multiple stochastic classifiers. We also investigate a memory management scheme and learning rate scheduling reflecting our proposed double perturbations. We demonstrate that our proposed method outperforms the state-of-the-art baseline methods by large margins on various TF-CL benchmarks.
△ Less
Submitted 18 February, 2024; v1 submitted 20 December, 2023;
originally announced December 2023.
-
Lennard Jones Token: a blockchain solution to scientific data curation
Authors:
Brian H. Lee,
Alejandro Strachan
Abstract:
Data science and artificial intelligence have become an indispensable part of scientific research. While such methods rely on high-quality and large quantities of machine-readable scientific data, the current scientific data infrastructure faces significant challenges that limit effective data curation and sharing. These challenges include insufficient return on investment for researchers to share…
▽ More
Data science and artificial intelligence have become an indispensable part of scientific research. While such methods rely on high-quality and large quantities of machine-readable scientific data, the current scientific data infrastructure faces significant challenges that limit effective data curation and sharing. These challenges include insufficient return on investment for researchers to share quality data, logistical difficulties in maintaining long-term data repositories, and the absence of standardized methods for evaluating the relative importance of various datasets. To address these issues, this paper presents the Lennard Jones Token, a blockchain-based proof-of-concept solution implemented on the Ethereum network. The token system incentivizes users to submit optimized structures of Lennard Jones particles by offering token rewards, while also charging for access to these valuable structures. Utilizing smart contracts, the system automates the evaluation of submitted data, ensuring that only structures with energies lower than those in the existing database for a given cluster size are rewarded. The paper explores the details of the Lennard Jones Token as a proof of concept and proposes future blockchain-based tokens aimed at enhancing the curation and sharing of scientific data.
△ Less
Submitted 1 December, 2023;
originally announced December 2023.
-
Online Continual Learning on Hierarchical Label Expansion
Authors:
Byung Hyun Lee,
Okchul Jung,
Jonghyun Choi,
Se Young Chun
Abstract:
Continual learning (CL) enables models to adapt to new tasks and environments without forgetting previously learned knowledge. While current CL setups have ignored the relationship between labels in the past task and the new task with or without small task overlaps, real-world scenarios often involve hierarchical relationships between old and new tasks, posing another challenge for traditional CL…
▽ More
Continual learning (CL) enables models to adapt to new tasks and environments without forgetting previously learned knowledge. While current CL setups have ignored the relationship between labels in the past task and the new task with or without small task overlaps, real-world scenarios often involve hierarchical relationships between old and new tasks, posing another challenge for traditional CL approaches. To address this challenge, we propose a novel multi-level hierarchical class incremental task configuration with an online learning constraint, called hierarchical label expansion (HLE). Our configuration allows a network to first learn coarse-grained classes, with data labels continually expanding to more fine-grained classes in various hierarchy depths. To tackle this new setup, we propose a rehearsal-based method that utilizes hierarchy-aware pseudo-labeling to incorporate hierarchical class information. Additionally, we propose a simple yet effective memory management and sampling strategy that selectively adopts samples of newly encountered classes. Our experiments demonstrate that our proposed method can effectively use hierarchy on our HLE setup to improve classification accuracy across all levels of hierarchies, regardless of depth and class imbalance ratio, outperforming prior state-of-the-art works by significant margins while also outperforming them on the conventional disjoint, blurry and i-Blurry CL setups.
△ Less
Submitted 28 August, 2023;
originally announced August 2023.
-
Efficient and Accurate Quantized Image Super-Resolution on Mobile NPUs, Mobile AI & AIM 2022 challenge: Report
Authors:
Andrey Ignatov,
Radu Timofte,
Maurizio Denna,
Abdel Younes,
Ganzorig Gankhuyag,
**gang Huh,
Myeong Kyun Kim,
Kihwan Yoon,
Hyeon-Cheol Moon,
Seungho Lee,
Yoonsik Choe,
**woo Jeong,
Sungjei Kim,
Maciej Smyl,
Tomasz Latkowski,
Pawel Kubik,
Michal Sokolski,
Yujie Ma,
Jiahao Chao,
Zhou Zhou,
Hongfan Gao,
Zhengfeng Yang,
Zhenbing Zeng,
Zhengyang Zhuge,
Chenghua Li
, et al. (71 additional authors not shown)
Abstract:
Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose…
▽ More
Image super-resolution is a common task on mobile and IoT devices, where one often needs to upscale and enhance low-resolution images and video frames. While numerous solutions have been proposed for this problem in the past, they are usually not compatible with low-power mobile NPUs having many computational and memory constraints. In this Mobile AI challenge, we address this problem and propose the participants to design an efficient quantized image super-resolution solution that can demonstrate a real-time performance on mobile NPUs. The participants were provided with the DIV2K dataset and trained INT8 models to do a high-quality 3X image upscaling. The runtime of all models was evaluated on the Synaptics VS680 Smart Home board with a dedicated edge NPU capable of accelerating quantized neural networks. All proposed solutions are fully compatible with the above NPU, demonstrating an up to 60 FPS rate when reconstructing Full HD resolution images. A detailed description of all models developed in the challenge is provided in this paper.
△ Less
Submitted 7 November, 2022;
originally announced November 2022.
-
Efficient Single-Image Depth Estimation on Mobile Devices, Mobile AI & AIM 2022 Challenge: Report
Authors:
Andrey Ignatov,
Grigory Malivenko,
Radu Timofte,
Lukasz Treszczotko,
Xin Chang,
Piotr Ksiazek,
Michal Lopuszynski,
Maciej Pioro,
Rafal Rudnicki,
Maciej Smyl,
Yujie Ma,
Zhenyu Li,
Zehui Chen,
Jialei Xu,
Xianming Liu,
Junjun Jiang,
XueChao Shi,
Difan Xu,
Yanan Li,
Xiaotao Wang,
Lei Lei,
Ziyu Zhang,
Yicheng Wang,
Zilong Huang,
Guozhong Luo
, et al. (14 additional authors not shown)
Abstract:
Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth es…
▽ More
Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth estimation solutions that can show a real-time performance on IoT platforms and smartphones. For this, the participants used a large-scale RGB-to-depth dataset that was collected with the ZED stereo camera capable to generated depth maps for objects located at up to 50 meters. The runtime of all models was evaluated on the Raspberry Pi 4 platform, where the developed solutions were able to generate VGA resolution depth maps at up to 27 FPS while achieving high fidelity results. All models developed in the challenge are also compatible with any Android or Linux-based mobile devices, their detailed description is provided in this paper.
△ Less
Submitted 7 November, 2022;
originally announced November 2022.
-
Multi-View Attention Transfer for Efficient Speech Enhancement
Authors:
Wooseok Shin,
Hyun Joon Park,
** Sob Kim,
Byung Hoon Lee,
Sung Won Han
Abstract:
Recent deep learning models have achieved high performance in speech enhancement; however, it is still challenging to obtain a fast and low-complexity model without significant performance degradation. Previous knowledge distillation studies on speech enhancement could not solve this problem because their output distillation methods do not fit the speech enhancement task in some aspects. In this s…
▽ More
Recent deep learning models have achieved high performance in speech enhancement; however, it is still challenging to obtain a fast and low-complexity model without significant performance degradation. Previous knowledge distillation studies on speech enhancement could not solve this problem because their output distillation methods do not fit the speech enhancement task in some aspects. In this study, we propose multi-view attention transfer (MV-AT), a feature-based distillation, to obtain efficient speech enhancement models in the time domain. Based on the multi-view features extraction model, MV-AT transfers multi-view knowledge of the teacher network to the student network without additional parameters. The experimental results show that the proposed method consistently improved the performance of student models of various sizes on the Valentini and deep noise suppression (DNS) datasets. MANNER-S-8.1GF with our proposed method, a lightweight model for efficient deployment, achieved 15.4x and 4.71x fewer parameters and floating-point operations (FLOPs), respectively, compared to the baseline model with similar performance.
△ Less
Submitted 30 October, 2022; v1 submitted 22 August, 2022;
originally announced August 2022.
-
Empirically Accelerating Scaled Gradient Projection Using Deep Neural Network For Inverse Problems In Image Processing
Authors:
Byung Hyun Lee,
Se Young Chun
Abstract:
Recently, deep neural networks (DNNs) have shown advantages in accelerating optimization algorithms. One approach is to unfold finite number of iterations of conventional optimization algorithms and to learn parameters in the algorithms. However, these are forward methods and are indeed neither iterative nor convergent. Here, we present a novel DNN-based convergent iterative algorithm that acceler…
▽ More
Recently, deep neural networks (DNNs) have shown advantages in accelerating optimization algorithms. One approach is to unfold finite number of iterations of conventional optimization algorithms and to learn parameters in the algorithms. However, these are forward methods and are indeed neither iterative nor convergent. Here, we present a novel DNN-based convergent iterative algorithm that accelerates conventional optimization algorithms. We train a DNN to yield parameters in scaled gradient projection method. So far, these parameters have been chosen heuristically, but have shown to be crucial for good empirical performance. In simulation results, the proposed method significantly improves the empirical convergence rate over conventional optimization methods for various large-scale inverse problems in image processing.
△ Less
Submitted 21 April, 2021; v1 submitted 6 February, 2019;
originally announced February 2019.
-
A Variational Feature Encoding Method of 3D Object for Probabilistic Semantic SLAM
Authors:
H. W. Yu,
B. H. Lee
Abstract:
This paper presents a feature encoding method of complex 3D objects for high-level semantic features. Recent approaches to object recognition methods become important for semantic simultaneous localization and map** (SLAM). However, there is a lack of consideration of the probabilistic observation model for 3D objects, as the shape of a 3D object basically follows a complex probability distribut…
▽ More
This paper presents a feature encoding method of complex 3D objects for high-level semantic features. Recent approaches to object recognition methods become important for semantic simultaneous localization and map** (SLAM). However, there is a lack of consideration of the probabilistic observation model for 3D objects, as the shape of a 3D object basically follows a complex probability distribution. Furthermore, since the mobile robot equipped with a range sensor observes only a single view, much information of the object shape is discarded. These limitations are the major obstacles to semantic SLAM and view-independent loop closure using 3D object shapes as features. In order to enable the numerical analysis for the Bayesian inference, we approximate the true observation model of 3D objects to tractable distributions. Since the observation likelihood can be obtained from the generative model, we formulate the true generative model for 3D object with the Bayesian networks. To capture these complex distributions, we apply a variational auto-encoder. To analyze the approximated distributions and encoded features, we perform classification with maximum likelihood estimation and shape retrieval.
△ Less
Submitted 30 August, 2018;
originally announced August 2018.
-
Traffic Convexity Aware Cellular Networks: A Vehicular Heavy User Perspective
Authors:
Taehyoung Shim,
Jihong Park,
Seung-Woo Ko,
Seong-Lyun Kim,
Beom Hee Lee,
** Gu Choi
Abstract:
Rampant mobile traffic increase in modern cellular networks is mostly caused by large-sized multimedia contents. Recent advancements in smart devices as well as radio access technologies promote the consumption of bulky content, even for people in moving vehicles, referred to as vehicular heavy users. In this article the emergence of vehicular heavy user traffic is observed by field experiments co…
▽ More
Rampant mobile traffic increase in modern cellular networks is mostly caused by large-sized multimedia contents. Recent advancements in smart devices as well as radio access technologies promote the consumption of bulky content, even for people in moving vehicles, referred to as vehicular heavy users. In this article the emergence of vehicular heavy user traffic is observed by field experiments conducted in 2012 and 2015 in Seoul, Korea. The experiments reveal that such traffic is becoming dominant, captured by the 8.62 times increase in vehicular heavy user traffic while the total traffic increased 3.04 times. To resolve this so-called vehicular heavy user problem (VHP), we propose a cell association algorithm that exploits user demand diversity for different velocities. This user traffic pattern is discovered first by our field trials, which is convex-shaped over velocity, i.e. walking user traffic is less than stationary or vehicular user traffic. As the VHP becomes severe, numerical evaluation verifies the proposed user convexity aware association outperforms a well-known load balancing association in practice, cell range expansion (CRE). In addition to the cell association, several complementary techniques are suggested in line with the technical trend toward 5G.
△ Less
Submitted 21 August, 2015; v1 submitted 12 January, 2015;
originally announced January 2015.
-
Mobility Control for Machine-to-Machine LTE Systems
Authors:
Beom Hee Lee,
Seong-Lyun Kim
Abstract:
In this paper, we propose an efficient mobility control algorithm for the downlink multi-cell orthogonal frequency division multiplexing access (OFDMA) system for co-channel interference reduction. It divides each cell into several areas. The mobile nodes in each area find their own optimal position according to their present location. Both the signal to interference plus noise ratio (SINR) and th…
▽ More
In this paper, we propose an efficient mobility control algorithm for the downlink multi-cell orthogonal frequency division multiplexing access (OFDMA) system for co-channel interference reduction. It divides each cell into several areas. The mobile nodes in each area find their own optimal position according to their present location. Both the signal to interference plus noise ratio (SINR) and the capacity for each node are increased by the proposed mobility control algorithm. Simulation results say that, even the frequency reuse factor (FRF) is equal to 1, the average capacity is improved after applying the mobility control algorithm, compared to existing partial frequency reuse (PFR) scheme.
△ Less
Submitted 25 January, 2011; v1 submitted 25 January, 2011;
originally announced January 2011.
