Perception Over Time: Temporal Dynamics for Robust Image Understanding
Authors:
Maryam Daniali,
Edward Kim
Abstract:
While deep learning surpasses human-level performance in narrow and specific vision tasks, it is fragile and over-confident in classification. For example, minor transformations in perspective, illumination, or object deformation in the image space can result in drastically different labeling, which is especially transparent via adversarial perturbations. On the other hand, human visual perception…
▽ More
While deep learning surpasses human-level performance in narrow and specific vision tasks, it is fragile and over-confident in classification. For example, minor transformations in perspective, illumination, or object deformation in the image space can result in drastically different labeling, which is especially transparent via adversarial perturbations. On the other hand, human visual perception is orders of magnitude more robust to changes in the input stimulus. But unfortunately, we are far from fully understanding and integrating the underlying mechanisms that result in such robust perception. In this work, we introduce a novel method of incorporating temporal dynamics into static image understanding. We describe a neuro-inspired method that decomposes a single image into a series of coarse-to-fine images that simulates how biological vision integrates information over time. Next, we demonstrate how our novel visual perception framework can utilize this information "over time" using a biologically plausible algorithm with recurrent units, and as a result, significantly improving its accuracy and robustness over standard CNNs. We also compare our proposed approach with state-of-the-art models and explicitly quantify our adversarial robustness properties through multiple ablation studies. Our quantitative and qualitative results convincingly demonstrate exciting and transformative improvements over the standard computer vision and deep learning architectures used today.
△ Less
Submitted 11 March, 2022;
originally announced March 2022.
The Selectivity and Competition of the Mind's Eye in Visual Perception
Authors:
Edward Kim,
Maryam Daniali,
Jocelyn Rego,
Garrett T. Kenyon
Abstract:
Research has shown that neurons within the brain are selective to certain stimuli. For example, the fusiform face area (FFA) region is known by neuroscientists to selectively activate when people see faces over non-face objects. However, the mechanisms by which the primary visual system directs information to the correct higher levels of the brain are currently unknown. In our work, we mimic sever…
▽ More
Research has shown that neurons within the brain are selective to certain stimuli. For example, the fusiform face area (FFA) region is known by neuroscientists to selectively activate when people see faces over non-face objects. However, the mechanisms by which the primary visual system directs information to the correct higher levels of the brain are currently unknown. In our work, we mimic several high-level neural mechanisms of perception by creating a novel computational model that incorporates lateral and top down feedback in the form of hierarchical competition. Not only do we show that these elements can help explain the information flow and selectivity of high level areas within the brain, we also demonstrate that these neural mechanisms provide the foundation of a novel classification framework that rivals traditional supervised learning in computer vision. Additionally, we present both quantitative and qualitative results that demonstrate that our generative framework is consistent with neurological themes and enables simple, yet robust category level classification.
△ Less
Submitted 23 March, 2021; v1 submitted 22 November, 2020;
originally announced November 2020.
