-
FRAG: Frequency Adapting Group for Diffusion Video Editing
Authors:
Sunjae Yoon,
Gwanhyeong Koo,
Geonwoo Kim,
Chang D. Yoo
Abstract:
In video editing, the hallmark of a quality edit lies in its consistent and unobtrusive adjustment. Modification, when integrated, must be smooth and subtle, preserving the natural flow and aligning seamlessly with the original vision. Therefore, our primary focus is on overcoming the current challenges in high quality edit to ensure that each edit enhances the final product without disrupting its…
▽ More
In video editing, the hallmark of a quality edit lies in its consistent and unobtrusive adjustment. Modification, when integrated, must be smooth and subtle, preserving the natural flow and aligning seamlessly with the original vision. Therefore, our primary focus is on overcoming the current challenges in high quality edit to ensure that each edit enhances the final product without disrupting its intended essence. However, quality deterioration such as blurring and flickering is routinely observed in recent diffusion video editing systems. We confirm that this deterioration often stems from high-frequency leak: the diffusion model fails to accurately synthesize high-frequency components during denoising process. To this end, we devise Frequency Adapting Group (FRAG) which enhances the video quality in terms of consistency and fidelity by introducing a novel receptive field branch to preserve high-frequency components during the denoising process. FRAG is performed in a model-agnostic manner without additional training and validates the effectiveness on video editing benchmarks (i.e., TGVE, DAVIS).
△ Less
Submitted 10 June, 2024;
originally announced June 2024.
-
Wavelet-Guided Acceleration of Text Inversion in Diffusion-Based Image Editing
Authors:
Gwanhyeong Koo,
Sunjae Yoon,
Chang D. Yoo
Abstract:
In the field of image editing, Null-text Inversion (NTI) enables fine-grained editing while preserving the structure of the original image by optimizing null embeddings during the DDIM sampling process. However, the NTI process is time-consuming, taking more than two minutes per image. To address this, we introduce an innovative method that maintains the principles of the NTI while accelerating th…
▽ More
In the field of image editing, Null-text Inversion (NTI) enables fine-grained editing while preserving the structure of the original image by optimizing null embeddings during the DDIM sampling process. However, the NTI process is time-consuming, taking more than two minutes per image. To address this, we introduce an innovative method that maintains the principles of the NTI while accelerating the image editing process. We propose the WaveOpt-Estimator, which determines the text optimization endpoint based on frequency characteristics. Utilizing wavelet transform analysis to identify the image's frequency characteristics, we can limit text optimization to specific timesteps during the DDIM sampling process. By adopting the Negative-Prompt Inversion (NPI) concept, a target prompt representing the original image serves as the initial text value for optimization. This approach maintains performance comparable to NTI while reducing the average editing time by over 80% compared to the NTI method. Our method presents a promising approach for efficient, high-quality image editing based on diffusion models.
△ Less
Submitted 18 January, 2024;
originally announced January 2024.
-
Neutral Editing Framework for Diffusion-based Video Editing
Authors:
Sunjae Yoon,
Gwanhyeong Koo,
Ji Woo Hong,
Chang D. Yoo
Abstract:
Text-conditioned image editing has succeeded in various types of editing based on a diffusion framework. Unfortunately, this success did not carry over to a video, which continues to be challenging. Existing video editing systems are still limited to rigid-type editing such as style transfer and object overlay. To this end, this paper proposes Neutral Editing (NeuEdit) framework to enable complex…
▽ More
Text-conditioned image editing has succeeded in various types of editing based on a diffusion framework. Unfortunately, this success did not carry over to a video, which continues to be challenging. Existing video editing systems are still limited to rigid-type editing such as style transfer and object overlay. To this end, this paper proposes Neutral Editing (NeuEdit) framework to enable complex non-rigid editing by changing the motion of a person/object in a video, which has never been attempted before. NeuEdit introduces a concept of `neutralization' that enhances a tuning-editing process of diffusion-based editing systems in a model-agnostic manner by leveraging input video and text without any other auxiliary aids (e.g., visual masks, video captions). Extensive experiments on numerous videos demonstrate adaptability and effectiveness of the NeuEdit framework. The website of our work is available here: https://neuedit.github.io
△ Less
Submitted 10 December, 2023;
originally announced December 2023.
-
SCANet: Scene Complexity Aware Network for Weakly-Supervised Video Moment Retrieval
Authors:
Sunjae Yoon,
Gwanhyeong Koo,
Dahyun Kim,
Chang D. Yoo
Abstract:
Video moment retrieval aims to localize moments in video corresponding to a given language query. To avoid the expensive cost of annotating the temporal moments, weakly-supervised VMR (wsVMR) systems have been studied. For such systems, generating a number of proposals as moment candidates and then selecting the most appropriate proposal has been a popular approach. These proposals are assumed to…
▽ More
Video moment retrieval aims to localize moments in video corresponding to a given language query. To avoid the expensive cost of annotating the temporal moments, weakly-supervised VMR (wsVMR) systems have been studied. For such systems, generating a number of proposals as moment candidates and then selecting the most appropriate proposal has been a popular approach. These proposals are assumed to contain many distinguishable scenes in a video as candidates. However, existing proposals of wsVMR systems do not respect the varying numbers of scenes in each video, where the proposals are heuristically determined irrespective of the video. We argue that the retrieval system should be able to counter the complexities caused by varying numbers of scenes in each video. To this end, we present a novel concept of a retrieval system referred to as Scene Complexity Aware Network (SCANet), which measures the `scene complexity' of multiple scenes in each video and generates adaptive proposals responding to variable complexities of scenes in each video. Experimental results on three retrieval benchmarks (i.e., Charades-STA, ActivityNet, TVR) achieve state-of-the-art performances and demonstrate the effectiveness of incorporating the scene complexity.
△ Less
Submitted 8 October, 2023;
originally announced October 2023.
-
Learning Disentangled Representations in Signed Directed Graphs without Social Assumptions
Authors:
Geonwoo Ko,
**hong Jung
Abstract:
Signed graphs are complex systems that represent trust relationships or preferences in various domains. Learning node representations in such graphs is crucial for many mining tasks. Although real-world signed relationships can be influenced by multiple latent factors, most existing methods often oversimplify the modeling of signed relationships by relying on social theories and treating them as s…
▽ More
Signed graphs are complex systems that represent trust relationships or preferences in various domains. Learning node representations in such graphs is crucial for many mining tasks. Although real-world signed relationships can be influenced by multiple latent factors, most existing methods often oversimplify the modeling of signed relationships by relying on social theories and treating them as simplistic factors. This limits their expressiveness and their ability to capture the diverse factors that shape these relationships. In this paper, we propose DINES, a novel method for learning disentangled node representations in signed directed graphs without social assumptions. We adopt a disentangled framework that separates each embedding into distinct factors, allowing for capturing multiple latent factors. We also explore lightweight graph convolutions that focus solely on sign and direction, without depending on social theories. Additionally, we propose a decoder that effectively classifies an edge's sign by considering correlations between the factors. To further enhance disentanglement, we jointly train a self-supervised factor discriminator with our encoder and decoder. Throughout extensive experiments on real-world signed directed graphs, we show that DINES effectively learns disentangled node representations, and significantly outperforms its competitors in the sign prediction task.
△ Less
Submitted 6 July, 2023;
originally announced July 2023.
-
INT2.1: Towards Fine-Tunable Quantized Large Language Models with Error Correction through Low-Rank Adaptation
Authors:
Yuji Chai,
John Gkountouras,
Glenn G. Ko,
David Brooks,
Gu-Yeon Wei
Abstract:
We introduce a method that dramatically reduces fine-tuning VRAM requirements and rectifies quantization errors in quantized Large Language Models. First, we develop an extremely memory-efficient fine-tuning (EMEF) method for quantized models using Low-Rank Adaptation (LoRA), and drawing upon it, we construct an error-correcting algorithm designed to minimize errors induced by the quantization pro…
▽ More
We introduce a method that dramatically reduces fine-tuning VRAM requirements and rectifies quantization errors in quantized Large Language Models. First, we develop an extremely memory-efficient fine-tuning (EMEF) method for quantized models using Low-Rank Adaptation (LoRA), and drawing upon it, we construct an error-correcting algorithm designed to minimize errors induced by the quantization process. Our method reduces the memory requirements by up to 5.6 times, which enables fine-tuning a 7 billion parameter Large Language Model (LLM) on consumer laptops. At the same time, we propose a Low-Rank Error Correction (LREC) method that exploits the added LoRA layers to ameliorate the gap between the quantized model and its float point counterpart. Our error correction framework leads to a fully functional INT2 quantized LLM with the capacity to generate coherent English text. To the best of our knowledge, this is the first INT2 Large Language Model that has been able to reach such a performance. The overhead of our method is merely a 1.05 times increase in model size, which translates to an effective precision of INT2.1. Also, our method readily generalizes to other quantization standards, such as INT3, INT4, and INT8, restoring their lost performance, which marks a significant milestone in the field of model quantization. The strategies delineated in this paper hold promising implications for the future development and optimization of quantized models, marking a pivotal shift in the landscape of low-resource machine learning computations.
△ Less
Submitted 13 June, 2023;
originally announced June 2023.
-
Satellite radio detection via dual-microwave Rydberg spectroscopy
Authors:
Peter K Elgee,
Joshua C Hill,
Kermit-James E Leblanc,
Gabriel D Ko,
Paul D Kunz,
David H Meyer,
Kevin C Cox
Abstract:
Rydberg electric field sensors exploit the large number of Rydberg resonances to provide sensitivity over a broad range of the electromagnetic spectrum. However, due to the difficulty of accessing resonant Rydberg states at ultra-high frequency (UHF) and below, ubiquitous bands in the world's current wireless communications infrastructure, they currently fall short in sensitivity in this range. We…
▽ More
Rydberg electric field sensors exploit the large number of Rydberg resonances to provide sensitivity over a broad range of the electromagnetic spectrum. However, due to the difficulty of accessing resonant Rydberg states at ultra-high frequency (UHF) and below, ubiquitous bands in the world's current wireless communications infrastructure, they currently fall short in sensitivity in this range. We present a resonant Rydberg electric field sensor operating in the UHF band using a dual-optical dual-microwave spectroscopy scheme. Adding an additional microwave photon allows us to access transitions between Rydberg states with higher angular momentum ($L = 3 \rightarrow 4$), which have lower resonant frequencies than transitions typically used in Rydberg sensors. We discuss the applicability of this type of sensor across the UHF band and below, and measure the resonant sensitivity of our system at 2.3 GHz to be 70(5) $μ$Vm$^{-1}\text{Hz}^{-1/2}$, 50 times better than the measured sensitivity with a far off-resonant probing scheme at this frequency. We also show the effectiveness of this sensing scheme by measuring Sirius XM satellite radio (2.320 - 2.345 GHz) received outside the laboratory and rebroadcast onto the atoms.
△ Less
Submitted 15 May, 2023;
originally announced May 2023.
-
Dynamical Billiard and a long-time behavior of the Boltzmann equation in general 3D toroidal domains
Authors:
Gyounghun Ko,
Chanwoo Kim,
Donghyun Lee
Abstract:
Establishing global well-posedness and convergence toward equilibrium of the Boltzmann equation with specular reflection boundary condition has been one of the central questions in the subject of kinetic theory. Despite recent significant progress in this question when domains are strictly convex, as shown by Guo and Kim-Lee, the same question without the strict convexity of domains is still total…
▽ More
Establishing global well-posedness and convergence toward equilibrium of the Boltzmann equation with specular reflection boundary condition has been one of the central questions in the subject of kinetic theory. Despite recent significant progress in this question when domains are strictly convex, as shown by Guo and Kim-Lee, the same question without the strict convexity of domains is still totally open in 3D. The major difficulty arises when a billiard map has an infinite number of bounces in a finite time interval or when the map fails to be Lipschitz continuous, both of which happen generically when the domain is non-convex. In this paper, we develop a new method to control a billiard map on a surface of revolution generated by revolving any planar analytic convex closed curve (e.g., typical shape of tokamak reactors' chamber). In particular, we classify and measure the size (to be small) of a pullback set (along the billiard trajectory) of the infinite-bouncing and singular-bouncing cases. As a consequence, we solve the open question affirmatively in such domains. To the best of our knowledge, this work is the first construction of global solutions to the hard-sphere Boltzmann equation in generic non-convex 3-dimensional domains. In Appendix, we introduce a novel method for constructive coercivity of a linearized collision operator $L$ when the specular boundary condition is imposed. In particular, this method works for a periodic cylindrical domain with an annulus cross-section.
△ Less
Submitted 8 July, 2023; v1 submitted 10 April, 2023;
originally announced April 2023.
-
Large amplitude problem of BGK model: Relaxation to quadratic nonlinearity
Authors:
Gi-Chan Bae,
Gyounghun Ko,
Donghyun Lee,
Seok-Bae Yun
Abstract:
Bhatnagar-Gross-Krook (BGK) equation is a relaxation model of the Boltzmann equation which is widely used in place of the Boltzmann equation for the simulation of various kinetic flow problems. In this work, we study the asymptotic stability of the BGK model when the initial data is not necessarily close to the global equilibrium pointwisely. Due to the highly nonlinear structure of the relaxation…
▽ More
Bhatnagar-Gross-Krook (BGK) equation is a relaxation model of the Boltzmann equation which is widely used in place of the Boltzmann equation for the simulation of various kinetic flow problems. In this work, we study the asymptotic stability of the BGK model when the initial data is not necessarily close to the global equilibrium pointwisely. Due to the highly nonlinear structure of the relaxation operator, the argument developed to derive the bootstrap estimate for the Boltzmann equation leads to a weaker estimate in the case of the BGK model, which does not exclude the possible blow-up of the perturbation. To overcome this issue, we carry out a refined analysis of the macroscopic fields to guarantee that the system transits from a highly nonlinear regime into a quadratic nonlinear regime after a long but finite time, in which the highly nonlinear perturbative term relaxes to essentially quadratic nonlinearity.
△ Less
Submitted 25 January, 2023; v1 submitted 24 January, 2023;
originally announced January 2023.
-
SpeedLimit: Neural Architecture Search for Quantized Transformer Models
Authors:
Yuji Chai,
Luke Bailey,
Yunho **,
Matthew Karle,
Glenn G. Ko,
David Brooks,
Gu-Yeon Wei,
H. T. Kung
Abstract:
While research in the field of transformer models has primarily focused on enhancing performance metrics such as accuracy and perplexity, practical applications in industry often necessitate a rigorous consideration of inference latency constraints. Addressing this challenge, we introduce SpeedLimit, a novel Neural Architecture Search (NAS) technique that optimizes accuracy whilst adhering to an u…
▽ More
While research in the field of transformer models has primarily focused on enhancing performance metrics such as accuracy and perplexity, practical applications in industry often necessitate a rigorous consideration of inference latency constraints. Addressing this challenge, we introduce SpeedLimit, a novel Neural Architecture Search (NAS) technique that optimizes accuracy whilst adhering to an upper-bound latency constraint. Our method incorporates 8-bit integer quantization in the search process to outperform the current state-of-the-art technique. Our results underline the feasibility and efficacy of seeking an optimal balance between performance and latency, providing new avenues for deploying state-of-the-art transformer models in latency-sensitive environments.
△ Less
Submitted 13 October, 2023; v1 submitted 24 September, 2022;
originally announced September 2022.
-
On $C^2$ solution of the free-transport equation in a disk
Authors:
Gyounghun Ko,
Donghyun Lee
Abstract:
The free transport operator of probability density function $f(t,x,v)$ is one the most fundamental operator which is widely used in many areas of PDE theory including kinetic theory, in particular. When it comes to general boundary problems in kinetic theory, however, it is well-known that high order regularity is very hard to obtain in general. In this paper, we study the free transport equation…
▽ More
The free transport operator of probability density function $f(t,x,v)$ is one the most fundamental operator which is widely used in many areas of PDE theory including kinetic theory, in particular. When it comes to general boundary problems in kinetic theory, however, it is well-known that high order regularity is very hard to obtain in general. In this paper, we study the free transport equation in a disk with the specular boundary condition. We obtain initial-boundary compatibility conditions for $C^1_{t,x,v}$ and $C^2_{t,x,v}$ regularity of the solution. We also provide regularity estimates.
△ Less
Submitted 12 September, 2022; v1 submitted 3 December, 2021;
originally announced December 2021.
-
Do photon-number-resolving detectors provide valid evidence for the discrete nature of light?
Authors:
Morgan C. Williamson,
Gabriel D. Ko,
Brian R. La Cour
Abstract:
We consider the question of whether photon-number-resolving (PNR) detectors provide compelling evidence for the discrete nature of light; i.e., whether they indicate the prior presence of a certain number of discrete photons. To answer this question, we reveal the insufficient signal-to-noise ratio (SNR) of existing PNR detectors, and propose an alternative interpretation for the analysis of PNR d…
▽ More
We consider the question of whether photon-number-resolving (PNR) detectors provide compelling evidence for the discrete nature of light; i.e., whether they indicate the prior presence of a certain number of discrete photons. To answer this question, we reveal the insufficient signal-to-noise ratio (SNR) of existing PNR detectors, and propose an alternative interpretation for the analysis of PNR detector output that is consistent with a wave picture of light and does not rely on the presumption of light particles. This interpretation is based on the aggregation of correlated or accidentally coincident detections within a given detector coincidence window. Our interpretation accounts for the arbitrary character of detector coincidence windows and includes connections to established treatments of intensity interferometers. To validate our interpretation, we performed an experiment on a multiplexed PNR detector and examined the dependence of photon number on the coincidence window via post-processing. These observations were then compared to a fully classical wave model based on amplitude threshold detection, and the results were found to be in excellent agreement. We find that results from low SNR PNR detectors, such as those existing in the literature, are able to be described by classical descriptions, and therefore do not demonstrate evidence for the discrete nature of light.
△ Less
Submitted 8 October, 2021;
originally announced October 2021.
-
Unsupervised Detection of Adversarial Examples with Model Explanations
Authors:
Gihyuk Ko,
Gyumin Lim
Abstract:
Deep Neural Networks (DNNs) have shown remarkable performance in a diverse range of machine learning applications. However, it is widely known that DNNs are vulnerable to simple adversarial perturbations, which causes the model to incorrectly classify inputs. In this paper, we propose a simple yet effective method to detect adversarial examples, using methods developed to explain the model's behav…
▽ More
Deep Neural Networks (DNNs) have shown remarkable performance in a diverse range of machine learning applications. However, it is widely known that DNNs are vulnerable to simple adversarial perturbations, which causes the model to incorrectly classify inputs. In this paper, we propose a simple yet effective method to detect adversarial examples, using methods developed to explain the model's behavior. Our key observation is that adding small, humanly imperceptible perturbations can lead to drastic changes in the model explanations, resulting in unusual or irregular forms of explanations. From this insight, we propose an unsupervised detection of adversarial examples using reconstructor networks trained only on model explanations of benign examples. Our evaluations with MNIST handwritten dataset show that our method is capable of detecting adversarial examples generated by the state-of-the-art algorithms with high confidence. To the best of our knowledge, this work is the first in suggesting unsupervised defense method using model explanations.
△ Less
Submitted 22 July, 2021;
originally announced July 2021.
-
The Virtual Quantum Optics Laboratory
Authors:
Brian R. La Cour,
Maria Maynard,
Parth Shroff,
Gabriel Ko,
Evan Ellis
Abstract:
We present a web-based software tool, the Virtual Quantum Optics Laboratory (VQOL), that may be used for designing and executing realistic simulations of quantum optics experiments. A graphical user interface allows one to rapidly build and configure a variety of different optical experiments, while the runtime environment provides unique capabilities for visualization and analysis. All standard l…
▽ More
We present a web-based software tool, the Virtual Quantum Optics Laboratory (VQOL), that may be used for designing and executing realistic simulations of quantum optics experiments. A graphical user interface allows one to rapidly build and configure a variety of different optical experiments, while the runtime environment provides unique capabilities for visualization and analysis. All standard linear optical components are available as well as sources of thermal, coherent, and entangled Gaussian states. A unique aspect of VQOL is the introduction of non-Gaussian measurements using detectors modeled as deterministic devices that "click" when the amplitude of the light falls above a given threshold. We describe the underlying theoretical models and provide several illustrative examples. We find that VQOL provides a a faithful representation of many experimental quantum optics phenomena and may serve as both a useful instructional tool for students as well as a valuable research tool for practitioners.
△ Less
Submitted 15 May, 2021;
originally announced May 2021.
-
The large amplitude solution of the Boltzmann equation with soft potential
Authors:
Gyounghun Ko,
Donghyun Lee,
Kwanghyuk Park
Abstract:
In this paper, we deal with (angular cut-off) Boltzmann equation with soft potential ($-3<γ<0$). In particular, we construct a unique global solution in $L^\infty_{x,v}$ which converges to global equilibrium asymptotically provided that initial data has a large amplitude but with sufficiently small relative entropy. Because frequency multiplier is not uniformly positive anymore, unlike hard potent…
▽ More
In this paper, we deal with (angular cut-off) Boltzmann equation with soft potential ($-3<γ<0$). In particular, we construct a unique global solution in $L^\infty_{x,v}$ which converges to global equilibrium asymptotically provided that initial data has a large amplitude but with sufficiently small relative entropy. Because frequency multiplier is not uniformly positive anymore, unlike hard potential case, time-involved velocity weight will be used to derive sub-exponential decay of the solution. Motivated by recent development of $L^2\mbox{-}L^\infty$ approach also, we introduce some modified estimates of quadratic nonlinear terms. Linearized collision kernel will be treated in a subtle manner to control singularity of soft potential kernel.
△ Less
Submitted 22 April, 2021; v1 submitted 20 April, 2021;
originally announced April 2021.
-
The Boltzmann equation with large-amplitude initial data and specular reflection boundary condition
Authors:
Renjun Duan,
Gyounghun Ko,
Donghyun Lee
Abstract:
For the Boltzmann equation with cutoff hard potentials, we construct the unique global solution converging with an exponential rate in large time to global Maxwellians not only for the specular reflection boundary condition with the bounded convex C^3 domain but also for a class of large amplitude initial data where the L^infty norm with a suitable velocity weight can be arbitrarily large but the…
▽ More
For the Boltzmann equation with cutoff hard potentials, we construct the unique global solution converging with an exponential rate in large time to global Maxwellians not only for the specular reflection boundary condition with the bounded convex C^3 domain but also for a class of large amplitude initial data where the L^infty norm with a suitable velocity weight can be arbitrarily large but the relative entropy need to be small. A key point in the proof is to introduce a delicate nonlinear iterative process of estimating the gain term basing on the triple Duhamel iteration along the linearized dynamics.
△ Less
Submitted 3 November, 2020;
originally announced November 2020.
-
Extending Class Activation Map** Using Gaussian Receptive Field
Authors:
Bum Jun Kim,
Gyogwon Koo,
Hyeyeon Choi,
Sang Woo Kim
Abstract:
This paper addresses the visualization task of deep learning models. To improve Class Activation Map** (CAM) based visualization method, we offer two options. First, we propose Gaussian upsampling, an improved upsampling method that can reflect the characteristics of deep learning models. Second, we identify and modify unnatural terms in the mathematical derivation of the existing CAM studies. B…
▽ More
This paper addresses the visualization task of deep learning models. To improve Class Activation Map** (CAM) based visualization method, we offer two options. First, we propose Gaussian upsampling, an improved upsampling method that can reflect the characteristics of deep learning models. Second, we identify and modify unnatural terms in the mathematical derivation of the existing CAM studies. Based on two options, we propose Extended-CAM, an advanced CAM-based visualization method, which exhibits improved theoretical properties. Experimental results show that Extended-CAM provides more accurate visualization than the existing methods.
△ Less
Submitted 15 January, 2020;
originally announced January 2020.
-
CHIPKIT: An agile, reusable open-source framework for rapid test chip development
Authors:
Paul Whatmough,
Marco Donato,
Glenn Ko,
Sae-Kyu Lee,
David Brooks,
Gu-Yeon Wei
Abstract:
The current trend for domain-specific architectures (DSAs) has led to renewed interest in research test chips to demonstrate new specialized hardware. Tape-outs also offer huge pedagogical value garnered from real hands-on exposure to the whole system stack. However, successful tape-outs demand hard-earned experience, and the design process is time consuming and fraught with challenges. Therefore,…
▽ More
The current trend for domain-specific architectures (DSAs) has led to renewed interest in research test chips to demonstrate new specialized hardware. Tape-outs also offer huge pedagogical value garnered from real hands-on exposure to the whole system stack. However, successful tape-outs demand hard-earned experience, and the design process is time consuming and fraught with challenges. Therefore, custom chips have remained the preserve of a small number of research groups, typically focused on circuit design research. This paper describes the CHIPKIT framework. We describe a reusable SoC subsystem which provides basic IO, an on-chip programmable host, memory and peripherals. This subsystem can be readily extended with new IP blocks to generate custom test chips. We also present an agile RTL development flow, including a code generation tool calledVGEN. Finally, we outline best practices for full-chip validation across the entire design cycle.
△ Less
Submitted 26 May, 2020; v1 submitted 13 January, 2020;
originally announced January 2020.
-
Low-Power Computer Vision: Status, Challenges, Opportunities
Authors:
Sergei Alyamkin,
Matthew Ardi,
Alexander C. Berg,
Achille Brighton,
Bo Chen,
Yiran Chen,
Hsin-Pai Cheng,
Zichen Fan,
Chen Feng,
Bo Fu,
Kent Gauen,
Abhinav Goel,
Alexander Goncharenko,
Xuyang Guo,
Soonhoi Ha,
Andrew Howard,
Xiao Hu,
Yuanjun Huang,
Donghyun Kang,
Jaeyoun Kim,
Jong Gook Ko,
Alexander Kondratyev,
Junhyeok Lee,
Seungjae Lee,
Suwoong Lee
, et al. (19 additional authors not shown)
Abstract:
Computer vision has achieved impressive progress in recent years. Meanwhile, mobile phones have become the primary computing platforms for millions of people. In addition to mobile phones, many autonomous systems rely on visual data for making decisions and some of these systems have limited energy (such as unmanned aerial vehicles also called drones and mobile robots). These systems rely on batte…
▽ More
Computer vision has achieved impressive progress in recent years. Meanwhile, mobile phones have become the primary computing platforms for millions of people. In addition to mobile phones, many autonomous systems rely on visual data for making decisions and some of these systems have limited energy (such as unmanned aerial vehicles also called drones and mobile robots). These systems rely on batteries and energy efficiency is critical. This article serves two main purposes: (1) Examine the state-of-the-art for low-power solutions to detect objects in images. Since 2015, the IEEE Annual International Low-Power Image Recognition Challenge (LPIRC) has been held to identify the most energy-efficient computer vision solutions. This article summarizes 2018 winners' solutions. (2) Suggest directions for research as well as opportunities for low-power computer vision.
△ Less
Submitted 15 April, 2019;
originally announced April 2019.
-
Selective Distillation of Weakly Annotated GTD for Vision-based Slab Identification System
Authors:
Sang Jun Lee,
Sang Woo Kim,
Wookyong Kwon,
Gyogwon Koo,
Jong Pil Yun
Abstract:
This paper proposes an algorithm for recognizing slab identification numbers in factory scenes. In the development of a deep-learning based system, manual labeling to make ground truth data (GTD) is an important but expensive task. Furthermore, the quality of GTD is closely related to the performance of a supervised learning algorithm. To reduce manual work in the labeling process, we generated we…
▽ More
This paper proposes an algorithm for recognizing slab identification numbers in factory scenes. In the development of a deep-learning based system, manual labeling to make ground truth data (GTD) is an important but expensive task. Furthermore, the quality of GTD is closely related to the performance of a supervised learning algorithm. To reduce manual work in the labeling process, we generated weakly annotated GTD by marking only character centroids. Whereas bounding-boxes for characters require at least a drag-and-drop operation or two clicks to annotate a character location, the weakly annotated GTD requires a single click to record a character location. The main contribution of this paper is on selective distillation to improve the quality of the weakly annotated GTD. Because manual GTD are usually generated by many people, it may contain personal bias or human error. To address this problem, the information in manual GTD is integrated and refined by selective distillation. In the process of selective distillation, a fully convolutional network is trained using the weakly annotated GTD, and its prediction maps are selectively used to revise locations and boundaries of semantic regions of characters in the initial GTD. The modified GTD are used in the main training stage, and a post-processing is conducted to retrieve text information. Experiments were thoroughly conducted on actual industry data collected at a steelmaking factory to demonstrate the effectiveness of the proposed method.
△ Less
Submitted 13 December, 2018; v1 submitted 9 October, 2018;
originally announced October 2018.
-
Proxy Non-Discrimination in Data-Driven Systems
Authors:
Anupam Datta,
Matt Fredrikson,
Gihyuk Ko,
Piotr Mardziel,
Shayak Sen
Abstract:
Machine learnt systems inherit biases against protected classes, historically disparaged groups, from training data. Usually, these biases are not explicit, they rely on subtle correlations discovered by training algorithms, and are therefore difficult to detect. We formalize proxy discrimination in data-driven systems, a class of properties indicative of bias, as the presence of protected class c…
▽ More
Machine learnt systems inherit biases against protected classes, historically disparaged groups, from training data. Usually, these biases are not explicit, they rely on subtle correlations discovered by training algorithms, and are therefore difficult to detect. We formalize proxy discrimination in data-driven systems, a class of properties indicative of bias, as the presence of protected class correlates that have causal influence on the system's output. We evaluate an implementation on a corpus of social datasets, demonstrating how to validate systems against these properties and to repair violations where they occur.
△ Less
Submitted 25 July, 2017;
originally announced July 2017.
-
Use Privacy in Data-Driven Systems: Theory and Experiments with Machine Learnt Programs
Authors:
Anupam Datta,
Matthew Fredrikson,
Gihyuk Ko,
Piotr Mardziel,
Shayak Sen
Abstract:
This paper presents an approach to formalizing and enforcing a class of use privacy properties in data-driven systems. In contrast to prior work, we focus on use restrictions on proxies (i.e. strong predictors) of protected information types. Our definition relates proxy use to intermediate computations that occur in a program, and identify two essential properties that characterize this behavior:…
▽ More
This paper presents an approach to formalizing and enforcing a class of use privacy properties in data-driven systems. In contrast to prior work, we focus on use restrictions on proxies (i.e. strong predictors) of protected information types. Our definition relates proxy use to intermediate computations that occur in a program, and identify two essential properties that characterize this behavior: 1) its result is strongly associated with the protected information type in question, and 2) it is likely to causally affect the final output of the program. For a specific instantiation of this definition, we present a program analysis technique that detects instances of proxy use in a model, and provides a witness that identifies which parts of the corresponding program exhibit the behavior. Recognizing that not all instances of proxy use of a protected information type are inappropriate, we make use of a normative judgment oracle that makes this inappropriateness determination for a given witness. Our repair algorithm uses the witness of an inappropriate proxy use to transform the model into one that provably does not exhibit proxy use, while avoiding changes that unduly affect classification accuracy. Using a corpus of social datasets, our evaluation shows that these algorithms are able to detect proxy use instances that would be difficult to find using existing techniques, and subsequently remove them while maintaining acceptable classification performance.
△ Less
Submitted 7 September, 2017; v1 submitted 22 May, 2017;
originally announced May 2017.
