-
Dictionary Learning with Accumulator Neurons
Authors:
Gavin Parpart,
Carlos Gonzalez,
Terrence C. Stewart,
Edward Kim,
Jocelyn Rego,
Andrew O'Brien,
Steven Nesbit,
Garrett T. Kenyon,
Yi**g Watkins
Abstract:
The Locally Competitive Algorithm (LCA) uses local competition between non-spiking leaky integrator neurons to infer sparse representations, allowing for potentially real-time execution on massively parallel neuromorphic architectures such as Intel's Loihi processor. Here, we focus on the problem of inferring sparse representations from streaming video using dictionaries of spatiotemporal features…
▽ More
The Locally Competitive Algorithm (LCA) uses local competition between non-spiking leaky integrator neurons to infer sparse representations, allowing for potentially real-time execution on massively parallel neuromorphic architectures such as Intel's Loihi processor. Here, we focus on the problem of inferring sparse representations from streaming video using dictionaries of spatiotemporal features optimized in an unsupervised manner for sparse reconstruction. Non-spiking LCA has previously been used to achieve unsupervised learning of spatiotemporal dictionaries composed of convolutional kernels from raw, unlabeled video. We demonstrate how unsupervised dictionary learning with spiking LCA (\hbox{S-LCA}) can be efficiently implemented using accumulator neurons, which combine a conventional leaky-integrate-and-fire (\hbox{LIF}) spike generator with an additional state variable that is used to minimize the difference between the integrated input and the spiking output. We demonstrate dictionary learning across a wide range of dynamical regimes, from graded to intermittent spiking, for inferring sparse representations of both static images drawn from the CIFAR database as well as video frames captured from a DVS camera. On a classification task that requires identification of the suite from a deck of cards being rapidly flipped through as viewed by a DVS camera, we find essentially no degradation in performance as the LCA model used to infer sparse spatiotemporal representations migrates from graded to spiking. We conclude that accumulator neurons are likely to provide a powerful enabling component of future neuromorphic hardware for implementing online unsupervised learning of spatiotemporal dictionaries optimized for sparse reconstruction of streaming video from event based DVS cameras.
△ Less
Submitted 30 May, 2022;
originally announced May 2022.
-
Deep Camera Obscura: An Image Restoration Pipeline for Lensless Pinhole Photography
Authors:
Joshua D. Rego,
Huai** Chen,
Shuai Li,
**wei Gu,
Suren Jayasuriya
Abstract:
The lensless pinhole camera is perhaps the earliest and simplest form of an imaging system using only a pinhole-sized aperture in place of a lens. They can capture an infinite depth-of-field and offer greater freedom from optical distortion over their lens-based counterparts. However, the inherent limitations of a pinhole system result in lower sharpness from blur caused by optical diffraction and…
▽ More
The lensless pinhole camera is perhaps the earliest and simplest form of an imaging system using only a pinhole-sized aperture in place of a lens. They can capture an infinite depth-of-field and offer greater freedom from optical distortion over their lens-based counterparts. However, the inherent limitations of a pinhole system result in lower sharpness from blur caused by optical diffraction and higher noise levels due to low light throughput of the small aperture, requiring very long exposure times to capture well-exposed images. In this paper, we explore an image restoration pipeline using deep learning and domain-knowledge of the pinhole system to enhance the pinhole image quality through a joint denoise and deblur approach. Our approach allows for more practical exposure times for hand-held photography and provides higher image quality, making it more suitable for daily photography compared to other lensless cameras while kee** size and cost low. This opens up the potential of pinhole cameras to be used in smaller devices, such as smartphones.
△ Less
Submitted 12 August, 2021;
originally announced August 2021.
-
The Selectivity and Competition of the Mind's Eye in Visual Perception
Authors:
Edward Kim,
Maryam Daniali,
Jocelyn Rego,
Garrett T. Kenyon
Abstract:
Research has shown that neurons within the brain are selective to certain stimuli. For example, the fusiform face area (FFA) region is known by neuroscientists to selectively activate when people see faces over non-face objects. However, the mechanisms by which the primary visual system directs information to the correct higher levels of the brain are currently unknown. In our work, we mimic sever…
▽ More
Research has shown that neurons within the brain are selective to certain stimuli. For example, the fusiform face area (FFA) region is known by neuroscientists to selectively activate when people see faces over non-face objects. However, the mechanisms by which the primary visual system directs information to the correct higher levels of the brain are currently unknown. In our work, we mimic several high-level neural mechanisms of perception by creating a novel computational model that incorporates lateral and top down feedback in the form of hierarchical competition. Not only do we show that these elements can help explain the information flow and selectivity of high level areas within the brain, we also demonstrate that these neural mechanisms provide the foundation of a novel classification framework that rivals traditional supervised learning in computer vision. Additionally, we present both quantitative and qualitative results that demonstrate that our generative framework is consistent with neurological themes and enables simple, yet robust category level classification.
△ Less
Submitted 23 March, 2021; v1 submitted 22 November, 2020;
originally announced November 2020.
-
UDC 2020 Challenge on Image Restoration of Under-Display Camera: Methods and Results
Authors:
Yuqian Zhou,
Michael Kwan,
Kyle Tolentino,
Neil Emerton,
Sehoon Lim,
Tim Large,
Lijiang Fu,
Zhihong Pan,
Baopu Li,
Qirui Yang,
Yihao Liu,
Jigang Tang,
Tao Ku,
Shibin Ma,
Bingnan Hu,
Jiarong Wang,
Densen Puthussery,
Hrishikesh P S,
Melvin Kuriakose,
Jiji C V,
Varun Sundar,
Sumanth Hegde,
Divya Kothandaraman,
Kaushik Mitra,
Akashdeep Jassal
, et al. (20 additional authors not shown)
Abstract:
This paper is the report of the first Under-Display Camera (UDC) image restoration challenge in conjunction with the RLQ workshop at ECCV 2020. The challenge is based on a newly-collected database of Under-Display Camera. The challenge tracks correspond to two types of display: a 4k Transparent OLED (T-OLED) and a phone Pentile OLED (P-OLED). Along with about 150 teams registered the challenge, ei…
▽ More
This paper is the report of the first Under-Display Camera (UDC) image restoration challenge in conjunction with the RLQ workshop at ECCV 2020. The challenge is based on a newly-collected database of Under-Display Camera. The challenge tracks correspond to two types of display: a 4k Transparent OLED (T-OLED) and a phone Pentile OLED (P-OLED). Along with about 150 teams registered the challenge, eight and nine teams submitted the results during the testing phase for each track. The results in the paper are state-of-the-art restoration performance of Under-Display Camera Restoration. Datasets and paper are available at https://yzhouas.github.io/projects/UDC/udc.html.
△ Less
Submitted 18 August, 2020;
originally announced August 2020.
-
Probabilistic Interpretation for Correntropy with Complex Data
Authors:
João P. F. Guimarães,
Aluisio I. R. Fontes,
Joilson B. A. Rego,
Allan de M. Martins
Abstract:
Recent studies have demonstrated that correntropy is an efficient tool for analyzing higher-order statistical moments in nonGaussian noise environments. Although it has been used with complex data, some adaptations were then necessary without deriving a generic form so that similarities between complex random variables can be aggregated. This paper presents a novel probabilistic interpretation for…
▽ More
Recent studies have demonstrated that correntropy is an efficient tool for analyzing higher-order statistical moments in nonGaussian noise environments. Although it has been used with complex data, some adaptations were then necessary without deriving a generic form so that similarities between complex random variables can be aggregated. This paper presents a novel probabilistic interpretation for correntropy using complex-valued data called complex correntropy. An analytical recursive solution for the maximum complex correntropy criterion (MCCC) is introduced as based on the fixedpoint solution. This technique is applied to a simple system identification case study, as the results demonstrate prominent advantages regarding the proposed cost function if compared to the complex recursive least squares (RLS) algorithm. By using such probabilistic interpretation, correntropy can be applied to solve several problems involving complex data in a more straightforward way.
△ Less
Submitted 15 June, 2016;
originally announced June 2016.
