-
DAS: A Deformable Attention to Capture Salient Information in CNNs
Authors:
Farzad Salajegheh,
Nader Asadi,
Soroush Saryazdi,
Sudhir Mudur
Abstract:
Convolutional Neural Networks (CNNs) excel in local spatial pattern recognition. For many vision tasks, such as object recognition and segmentation, salient information is also present outside CNN's kernel boundaries. However, CNNs struggle in capturing such relevant information due to their confined receptive fields. Self-attention can improve a model's access to global information but increases…
▽ More
Convolutional Neural Networks (CNNs) excel in local spatial pattern recognition. For many vision tasks, such as object recognition and segmentation, salient information is also present outside CNN's kernel boundaries. However, CNNs struggle in capturing such relevant information due to their confined receptive fields. Self-attention can improve a model's access to global information but increases computational overhead. We present a fast and simple fully convolutional method called DAS that helps focus attention on relevant information. It uses deformable convolutions for the location of pertinent image regions and separable convolutions for efficiency. DAS plugs into existing CNNs and propagates relevant information using a gating mechanism. Compared to the O(n^2) computational complexity of transformer-style attention, DAS is O(n). Our claim is that DAS's ability to pay increased attention to relevant features results in performance improvements when added to popular CNNs for Image Classification and Object Detection. For example, DAS yields an improvement on Stanford Dogs (4.47%), ImageNet (1.91%), and COCO AP (3.3%) with base ResNet50 backbone. This outperforms other CNN attention mechanisms while using similar or less FLOPs. Our code will be publicly available.
△ Less
Submitted 20 November, 2023;
originally announced November 2023.
-
ConceptFusion: Open-set Multimodal 3D Map**
Authors:
Krishna Murthy Jatavallabhula,
Alihusein Kuwajerwala,
Qiao Gu,
Mohd Omama,
Tao Chen,
Alaa Maalouf,
Shuang Li,
Ganesh Iyer,
Soroush Saryazdi,
Nikhil Keetha,
Ayush Tewari,
Joshua B. Tenenbaum,
Celso Miguel de Melo,
Madhava Krishna,
Liam Paull,
Florian Shkurti,
Antonio Torralba
Abstract:
Building 3D maps of the environment is central to robot navigation, planning, and interaction with objects in a scene. Most existing approaches that integrate semantic concepts with 3D maps largely remain confined to the closed-set setting: they can only reason about a finite set of concepts, pre-defined at training time. Further, these maps can only be queried using class labels, or in recent wor…
▽ More
Building 3D maps of the environment is central to robot navigation, planning, and interaction with objects in a scene. Most existing approaches that integrate semantic concepts with 3D maps largely remain confined to the closed-set setting: they can only reason about a finite set of concepts, pre-defined at training time. Further, these maps can only be queried using class labels, or in recent work, using text prompts.
We address both these issues with ConceptFusion, a scene representation that is (1) fundamentally open-set, enabling reasoning beyond a closed set of concepts and (ii) inherently multimodal, enabling a diverse range of possible queries to the 3D map, from language, to images, to audio, to 3D geometry, all working in concert. ConceptFusion leverages the open-set capabilities of today's foundation models pre-trained on internet-scale data to reason about concepts across modalities such as natural language, images, and audio. We demonstrate that pixel-aligned open-set features can be fused into 3D maps via traditional SLAM and multi-view fusion approaches. This enables effective zero-shot spatial reasoning, not needing any additional training or finetuning, and retains long-tailed concepts better than supervised approaches, outperforming them by more than 40% margin on 3D IoU. We extensively evaluate ConceptFusion on a number of real-world datasets, simulated home environments, a real-world tabletop manipulation task, and an autonomous driving platform. We showcase new avenues for blending foundation models with 3D open-set multimodal map**.
For more information, visit our project page https://concept-fusion.github.io or watch our 5-minute explainer video https://www.youtube.com/watch?v=rkXgws8fiDs
△ Less
Submitted 23 October, 2023; v1 submitted 14 February, 2023;
originally announced February 2023.
-
gradSLAM: Automagically differentiable SLAM
Authors:
Krishna Murthy Jatavallabhula,
Soroush Saryazdi,
Ganesh Iyer,
Liam Paull
Abstract:
Blending representation learning approaches with simultaneous localization and map** (SLAM) systems is an open question, because of their highly modular and complex nature. Functionally, SLAM is an operation that transforms raw sensor inputs into a distribution over the state(s) of the robot and the environment. If this transformation (SLAM) were expressible as a differentiable function, we coul…
▽ More
Blending representation learning approaches with simultaneous localization and map** (SLAM) systems is an open question, because of their highly modular and complex nature. Functionally, SLAM is an operation that transforms raw sensor inputs into a distribution over the state(s) of the robot and the environment. If this transformation (SLAM) were expressible as a differentiable function, we could leverage task-based error signals to learn representations that optimize task performance. However, several components of a typical dense SLAM system are non-differentiable. In this work, we propose gradSLAM, a methodology for posing SLAM systems as differentiable computational graphs, which unifies gradient-based learning and SLAM. We propose differentiable trust-region optimizers, surface measurement and fusion schemes, and raycasting, without sacrificing accuracy. This amalgamation of dense SLAM with computational graphs enables us to backprop all the way from 3D maps to 2D pixels, opening up new possibilities in gradient-based learning for SLAM.
TL;DR: We leverage the power of automatic differentiation frameworks to make dense SLAM differentiable.
△ Less
Submitted 19 November, 2020; v1 submitted 23 October, 2019;
originally announced October 2019.
-
EDIZ: An Error Diffusion Image Zooming Scheme
Authors:
Soroush Saryazdi,
Saman Saryazdi,
Hossein Nezamabadi-pour
Abstract:
Interpolation based image zooming methods provide a high execution speed and low computational complexity. However, the quality of the zoomed images is unsatisfactory in many cases. The main challenge of super- resolution methods is to create new details to the image. This paper proposes a new algorithm to create new details using a zoom-out-zoom-in strategy. This strategy permits reducing blurrin…
▽ More
Interpolation based image zooming methods provide a high execution speed and low computational complexity. However, the quality of the zoomed images is unsatisfactory in many cases. The main challenge of super- resolution methods is to create new details to the image. This paper proposes a new algorithm to create new details using a zoom-out-zoom-in strategy. This strategy permits reducing blurring effects by adding the estimated error to the final image. Experimental results for natural images confirm the algorithm's ability to create visually pleasing results.
△ Less
Submitted 3 December, 2017;
originally announced December 2017.
-
NPC: Neighbors Progressive Competition Algorithm for Classification of Imbalanced Data Sets
Authors:
Soroush Saryazdi,
Bahareh Nikpour,
Hossein Nezamabadi-pour
Abstract:
Learning from many real-world datasets is limited by a problem called the class imbalance problem. A dataset is imbalanced when one class (the majority class) has significantly more samples than the other class (the minority class). Such datasets cause typical machine learning algorithms to perform poorly on the classification task. To overcome this issue, this paper proposes a new approach Neighb…
▽ More
Learning from many real-world datasets is limited by a problem called the class imbalance problem. A dataset is imbalanced when one class (the majority class) has significantly more samples than the other class (the minority class). Such datasets cause typical machine learning algorithms to perform poorly on the classification task. To overcome this issue, this paper proposes a new approach Neighbors Progressive Competition (NPC) for classification of imbalanced datasets. Whilst the proposed algorithm is inspired by weighted k-Nearest Neighbor (k-NN) algorithms, it has major differences from them. Unlike k- NN, NPC does not limit its decision criteria to a preset number of nearest neighbors. In contrast, NPC considers progressively more neighbors of the query sample in its decision making until the sum of grades for one class is much higher than the other classes. Furthermore, NPC uses a novel method for grading the training samples to compensate for the imbalance issue. The grades are calculated using both local and global information. In brief, the contribution of this paper is an entirely new classifier for handling the imbalance issue effectively without any manually-set parameters or any need for expert knowledge. Experimental results compare the proposed approach with five representative algorithms applied to fifteen imbalanced datasets and illustrate this algorithms effectiveness.
△ Less
Submitted 29 November, 2017;
originally announced November 2017.
-
Eye detection in digital images: challenges and solutions
Authors:
Mitra Montazeri,
Mahdieh Montazeri,
Saeid Saryazdi
Abstract:
Eye Detection has an important role in the field of biometric identification and known as one method of person's identification. In recent years, many efforts have been done which can detect eye automatically and with different image conditions. However, each method has its own drawbacks which can control some of these conditions. In this paper, different methods of eye detection will be categoriz…
▽ More
Eye Detection has an important role in the field of biometric identification and known as one method of person's identification. In recent years, many efforts have been done which can detect eye automatically and with different image conditions. However, each method has its own drawbacks which can control some of these conditions. In this paper, different methods of eye detection will be categorized and explained. In each category, the advantages and disadvantages of each method will be presented.
△ Less
Submitted 24 January, 2016; v1 submitted 19 January, 2016;
originally announced January 2016.
-
Combined A*-Ants Algorithm: A New Multi-Parameter Vehicle Navigation Scheme
Authors:
Hojjat Salehinejad,
Hossein Nezamabadi-pour,
Saeid Saryazdi,
Fereydoun Farrahi-Moghaddam
Abstract:
In this paper a multi-parameter A*(A- star)-ants based algorithm is proposed in order to find the best optimized multi-parameter path between two desired points in regions. This algorithm recognizes paths, according to user desired parameters using electronic maps. The proposed algorithm is a combination of A* and ants algorithm in which the proposed A* algorithm is the prologue to the suggested a…
▽ More
In this paper a multi-parameter A*(A- star)-ants based algorithm is proposed in order to find the best optimized multi-parameter path between two desired points in regions. This algorithm recognizes paths, according to user desired parameters using electronic maps. The proposed algorithm is a combination of A* and ants algorithm in which the proposed A* algorithm is the prologue to the suggested ant based algorithm .In fact, this A* algorithm invigorates some paths pheromones in ants algorithm. As one of implementations of this method, this algorithm was applied on a part of Kerman city, Iran as a multi-parameter vehicle navigator. It finds the best optimized multi-parameter direction between two desired junctions based on city traveler parameters. Comparison results between the proposed method and ants algorithm demonstrates efficiency and lower cost function results of the proposed method versus ants algorithm.
△ Less
Submitted 27 April, 2015;
originally announced April 2015.
