-
Region-Based Representations Revisited
Authors:
Michal Shlapentokh-Rothman,
Ansel Blume,
Yao Xiao,
Yuqun Wu,
Sethuraman T V,
Heyi Tao,
Jae Yong Lee,
Wilfredo Torres,
Yu-Xiong Wang,
Derek Hoiem
Abstract:
We investigate whether region-based representations are effective for recognition. Regions were once a mainstay in recognition approaches, but pixel and patch-based features are now used almost exclusively. We show that recent class-agnostic segmenters like SAM can be effectively combined with strong unsupervised representations like DINOv2 and used for a wide variety of tasks, including semantic…
▽ More
We investigate whether region-based representations are effective for recognition. Regions were once a mainstay in recognition approaches, but pixel and patch-based features are now used almost exclusively. We show that recent class-agnostic segmenters like SAM can be effectively combined with strong unsupervised representations like DINOv2 and used for a wide variety of tasks, including semantic segmentation, object-based image retrieval, and multi-image analysis. Once the masks and features are extracted, these representations, even with linear decoders, enable competitive performance, making them well suited to applications that require custom queries. The compactness of the representation also makes it well-suited to video analysis and other problems requiring inference across many images.
△ Less
Submitted 9 June, 2024; v1 submitted 4 February, 2024;
originally announced February 2024.
-
High-Throughput Virtual Screening of Small Molecule Inhibitors for SARS-CoV-2 Protein Targets with Deep Fusion Models
Authors:
Garrett A. Stevenson,
Derek Jones,
Hyo** Kim,
W. F. Drew Bennett,
Brian J. Bennion,
Monica Borucki,
Feliza Bourguet,
Aidan Epstein,
Magdalena Franco,
Brooke Harmon,
Stewart He,
Max P. Katz,
Daniel Kirshner,
Victoria Lao,
Edmond Y. Lau,
Jacky Lo,
Kevin McLoughlin,
Richard Mosesso,
Deepa K. Murugesh,
Oscar A. Negrete,
Edwin A. Saada,
Brent Segelke,
Maxwell Stefan,
Marisa W. Torres,
Dina Weilhammer
, et al. (7 additional authors not shown)
Abstract:
Structure-based Deep Fusion models were recently shown to outperform several physics- and machine learning-based protein-ligand binding affinity prediction methods. As part of a multi-institutional COVID-19 pandemic response, over 500 million small molecules were computationally screened against four protein structures from the novel coronavirus (SARS-CoV-2), which causes COVID-19. Three enhanceme…
▽ More
Structure-based Deep Fusion models were recently shown to outperform several physics- and machine learning-based protein-ligand binding affinity prediction methods. As part of a multi-institutional COVID-19 pandemic response, over 500 million small molecules were computationally screened against four protein structures from the novel coronavirus (SARS-CoV-2), which causes COVID-19. Three enhancements to Deep Fusion were made in order to evaluate more than 5 billion docked poses on SARS-CoV-2 protein targets. First, the Deep Fusion concept was refined by formulating the architecture as one, coherently backpropagated model (Coherent Fusion) to improve binding-affinity prediction accuracy. Secondly, the model was trained using a distributed, genetic hyper-parameter optimization. Finally, a scalable, high-throughput screening capability was developed to maximize the number of ligands evaluated and expedite the path to experimental evaluation. In this work, we present both the methods developed for machine learning-based high-throughput screening and results from using our computational pipeline to find SARS-CoV-2 inhibitors.
△ Less
Submitted 31 May, 2021; v1 submitted 9 April, 2021;
originally announced April 2021.
-
Constructing Locally Dense Point Clouds Using OpenSfM and ORB-SLAM2
Authors:
Fouad Amer,
Zixu Zhao,
Siwei Tang,
Wilfredo Torres
Abstract:
This paper aims at finding a method to register two different point clouds constructed by ORB-SLAM2 and OpenSfM. To do this, we post some tags with unique textures in the scene and take videos and photos of that area. Then we take short videos of only the tags to extract their features. By matching the ORB feature of the tags with their corresponding features in the scene, it is then possible to l…
▽ More
This paper aims at finding a method to register two different point clouds constructed by ORB-SLAM2 and OpenSfM. To do this, we post some tags with unique textures in the scene and take videos and photos of that area. Then we take short videos of only the tags to extract their features. By matching the ORB feature of the tags with their corresponding features in the scene, it is then possible to localize the position of these tags both in point clouds constructed by ORB-SLAM2 and OpenSfM. Thus, the best transformation matrix between two point clouds can be calculated, and the two point clouds can be aligned.
△ Less
Submitted 23 April, 2018;
originally announced April 2018.
-
Identification of Seed Cells in Multispectral Images for GrowCut Segmentation
Authors:
Wuilan Torres,
Antonio Rueda-Toicen
Abstract:
The segmentation of satellite images is a necessary step to perform object-oriented image classification, which has become relevant due to its applicability on images with a high spatial resolution. To perform object-oriented image classification, the studied image must first be segmented in uniform regions. This segmentation requires manual work by an expert user, who must exhaustively explore th…
▽ More
The segmentation of satellite images is a necessary step to perform object-oriented image classification, which has become relevant due to its applicability on images with a high spatial resolution. To perform object-oriented image classification, the studied image must first be segmented in uniform regions. This segmentation requires manual work by an expert user, who must exhaustively explore the image to establish thresholds that generate useful and representative segments without oversegmenting and without discarding representative segments. We propose a technique that automatically segments the multispectral image while facing these issues. We identify in the image homogenous zones according to their spectral signatures through the use of morphological filters. These homogenous zones are representatives of different types of land coverings in the image and are used as seeds for the GrowCut multispectral segmentation algorithm. GrowCut is a cellular automaton with competitive region growth, its cells are linked to every pixel in the image through three parameters: the pixel's spectral signature, a label, and a strength factor that represents the strength with which a cell defends its label. The seed cells possess maximum strength and maintain their state throughout the automaton's evolution. Starting from seed cells, each cell in the image is iteratively attacked by its neighboring cells. When the automaton stops updating its states, we obtain a segmented image where each pixel has taken the label of one of its cells. In this paper the algorithm was applied in an image acquired by Landsat8 on agricultural land of Calabozo, Guarico, Venezuela where there are different types of land coverings: agriculture, urban regions, water bodies, and savannas with different degrees of human intervention. The segmentation obtained is presented as irregular polygons enclosing geographical objects.
△ Less
Submitted 16 January, 2018;
originally announced January 2018.
-
Unsupervised Segmentation of Multispectral Images with Cellular Automata
Authors:
Wuilian Torres,
Antonio Rueda-Toicen
Abstract:
Multispectral images acquired by satellites are used to study phenomena on the Earth's surface. Unsupervised classification techniques analyze multispectral image content without considering prior knowledge of the observed terrain; this is done using techniques which group pixels that have similar statistics of digital level distribution in the various image channels. In this paper, we propose a m…
▽ More
Multispectral images acquired by satellites are used to study phenomena on the Earth's surface. Unsupervised classification techniques analyze multispectral image content without considering prior knowledge of the observed terrain; this is done using techniques which group pixels that have similar statistics of digital level distribution in the various image channels. In this paper, we propose a methodology for unsupervised classification based on a deterministic cellular automaton. The automaton is initialized in an unsupervised manner by setting seed cells, selected according to two criteria: to be representative of the spatial distribution of the dominant elements in the image, and to take into account the diversity of spectral signatures in the image. The automaton's evolution is based on an attack rule that is applied simultaneously to all its cells. Among the noteworthy advantages of deterministic cellular automata for multispectral processing of satellite imagery is the consideration of topological information in the image via seed positioning, and the ability to modify the scale of the study.
△ Less
Submitted 23 January, 2015;
originally announced January 2015.
