Showing 1–6 of 6 results for author: JaJa, J F

TAG: Boosting Text-VQA via Text-aware Visual Question-answer Generation

Authors: Jun Wang, Mingfei Gao, Yuqian Hu, Ramprasaath R. Selvaraju, Chetan Ramaiah, Ran Xu, Joseph F. JaJa, Larry S. Davis

Abstract: Text-VQA aims at answering questions that require understanding the textual cues in an image. Despite the great progress of existing Text-VQA methods, their performance suffers from insufficient human-labeled question-answer (QA) pairs. However, we observe that, in general, the scene text is not fully exploited in the existing datasets -- only a small portion of the text in each image participates… ▽ More Text-VQA aims at answering questions that require understanding the textual cues in an image. Despite the great progress of existing Text-VQA methods, their performance suffers from insufficient human-labeled question-answer (QA) pairs. However, we observe that, in general, the scene text is not fully exploited in the existing datasets -- only a small portion of the text in each image participates in the annotated QA activities. This results in a huge waste of useful information. To address this deficiency, we develop a new method to generate high-quality and diverse QA pairs by explicitly utilizing the existing rich text available in the scene context of each image. Specifically, we propose, TAG, a text-aware visual question-answer generation architecture that learns to produce meaningful, and accurate QA samples using a multimodal transformer. The architecture exploits underexplored scene text information and enhances scene understanding of Text-VQA models by combining the generated QA pairs with the initial training data. Extensive experimental results on two well-known Text-VQA benchmarks (TextVQA and ST-VQA) demonstrate that our proposed TAG effectively enlarges the training data that helps improve the Text-VQA performance without extra labeling effort. Moreover, our model outperforms state-of-the-art approaches that are pre-trained with extra large-scale data. Code is available at https://github.com/HenryJunW/TAG. △ Less

Submitted 7 October, 2022; v1 submitted 2 August, 2022; originally announced August 2022.

Comments: BMVC 2022

Learning Graph-Level Representations with Recurrent Neural Networks

Authors: Yu **, Joseph F. JaJa

Abstract: Recently a variety of methods have been developed to encode graphs into low-dimensional vectors that can be easily exploited by machine learning algorithms. The majority of these methods start by embedding the graph nodes into a low-dimensional vector space, followed by using some scheme to aggregate the node embeddings. In this work, we develop a new approach to learn graph-level representations,… ▽ More Recently a variety of methods have been developed to encode graphs into low-dimensional vectors that can be easily exploited by machine learning algorithms. The majority of these methods start by embedding the graph nodes into a low-dimensional vector space, followed by using some scheme to aggregate the node embeddings. In this work, we develop a new approach to learn graph-level representations, which includes a combination of unsupervised and supervised learning components. We start by learning a set of node representations in an unsupervised fashion. Graph nodes are mapped into node sequences sampled from random walk approaches approximated by the Gumbel-Softmax distribution. Recurrent neural network (RNN) units are modified to accommodate both the node representations as well as their neighborhood information. Experiments on standard graph classification benchmarks demonstrate that our proposed approach achieves superior or comparable performance relative to the state-of-the-art algorithms in terms of convergence speed and classification accuracy. We further illustrate the effectiveness of the different components used by our approach. △ Less

Submitted 11 September, 2018; v1 submitted 19 May, 2018; originally announced May 2018.

Comments: Submit to AAAI 2019

A High Performance Implementation of Spectral Clustering on CPU-GPU Platforms

Authors: Yu **, Joseph F. JaJa

Abstract: Spectral clustering is one of the most popular graph clustering algorithms, which achieves the best performance for many scientific and engineering applications. However, existing implementations in commonly used software platforms such as Matlab and Python do not scale well for many of the emerging Big Data applications. In this paper, we present a fast implementation of the spectral clustering a… ▽ More Spectral clustering is one of the most popular graph clustering algorithms, which achieves the best performance for many scientific and engineering applications. However, existing implementations in commonly used software platforms such as Matlab and Python do not scale well for many of the emerging Big Data applications. In this paper, we present a fast implementation of the spectral clustering algorithm on a CPU-GPU heterogeneous platform. Our implementation takes advantage of the computational power of the multi-core CPU and the massive multithreading and SIMD capabilities of GPUs. Given the input as data points in high dimensional space, we propose a parallel scheme to build a sparse similarity graph represented in a standard sparse representation format. Then we compute the smallest $k$ eigenvectors of the Laplacian matrix by utilizing the reverse communication interfaces of ARPACK software and cuSPARSE library, where $k$ is typically very large. Moreover, we implement a very fast parallelized $k$-means algorithm on GPUs. Our implementation is shown to be significantly faster compared to the best known Matlab and Python implementations for each step. In addition, our algorithm scales to problems with a very large number of clusters. △ Less

Submitted 12 February, 2018; originally announced February 2018.

Comments: 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (Parallel Computing and Optimization (PCO) workshop). Codes are available on https://github.com/yuj-umd/fastsc

Graph Coarsening with Preserved Spectral Properties

Authors: Yu **, Andreas Loukas, Joseph F. JaJa

Abstract: Large-scale graphs are widely used to represent object relationships in many real world applications. The occurrence of large-scale graphs presents significant computational challenges to process, analyze, and extract information. Graph coarsening techniques are commonly used to reduce the computational load while attempting to maintain the basic structural properties of the original graph. As the… ▽ More Large-scale graphs are widely used to represent object relationships in many real world applications. The occurrence of large-scale graphs presents significant computational challenges to process, analyze, and extract information. Graph coarsening techniques are commonly used to reduce the computational load while attempting to maintain the basic structural properties of the original graph. As there is no consensus on the specific graph properties preserved by coarse graphs, how to measure the differences between original and coarse graphs remains a key challenge. In this work, we introduce a new perspective regarding the graph coarsening based on concepts from spectral graph theory. We propose and justify new distance functions that characterize the differences between original and coarse graphs. We show that the proposed spectral distance naturally captures the structural differences in the graph coarsening process. In addition, we provide efficient graph coarsening algorithms to generate graphs which provably preserve the spectral properties from original graphs. Experiments show that our proposed algorithms consistently achieve better results compared to previous graph coarsening methods on graph classification and block recovery tasks. △ Less

Submitted 10 October, 2019; v1 submitted 12 February, 2018; originally announced February 2018.

Comments: Submitted to AISTATS 2020

A Data-Driven Approach to Extract Connectivity Structures from Diffusion Tensor Imaging Data

Authors: Yu **, Joseph F. JaJa, Rong Chen, Edward H. Herskovits

Abstract: Diffusion Tensor Imaging (DTI) is an effective tool for the analysis of structural brain connectivity in normal development and in a broad range of brain disorders. However efforts to derive inherent characteristics of structural brain networks have been hampered by the very high dimensionality of the data, relatively small sample sizes, and the lack of widely acceptable connectivity-based regions… ▽ More Diffusion Tensor Imaging (DTI) is an effective tool for the analysis of structural brain connectivity in normal development and in a broad range of brain disorders. However efforts to derive inherent characteristics of structural brain networks have been hampered by the very high dimensionality of the data, relatively small sample sizes, and the lack of widely acceptable connectivity-based regions of interests (ROIs). Typical approaches have focused either on regions defined by standard anatomical atlases that do not incorporate anatomical connectivity, or have been based on voxel-wise analysis, which results in loss of statistical power relative to structure-wise connectivity analysis. In this work, we propose a novel, computationally efficient iterative clustering method to generate connectivity-based whole-brain parcellations that converge to a stable parcellation in a few iterations. Our algorithm is based on a sparse representation of the whole brain connectivity matrix, which reduces the number of edges from around a half billion to a few million while incorporating the necessary spatial constraints. We show that the resulting regions in a sense capture the inherent connectivity information present in the data, and are stable with respect to initialization and the randomization scheme within the algorithm. These parcellations provide consistent structural regions across the subjects of population samples that are homogeneous with respect to anatomic connectivity. Our method also derives connectivity structures that can be used to distinguish between population samples with known different structural connectivity. In particular, new results in structural differences for different population samples such as Females vs Males, Normal Controls vs Schizophrenia, and different age groups in Normal Controls are also shown. △ Less

Submitted 12 February, 2018; originally announced February 2018.

Comments: Proceedings of 2015 IEEE International Conference on Big Data

Scalable Algorithms for Generating and Analyzing Structural Brain Networks with a Varying Number of Nodes

Authors: Yu **, Joseph F. JaJa, Rong Chen, Edward H. Herskovits

Abstract: Diffusion Magnetic Resonance Imaging (MRI) exploits the anisotropic diffusion of water molecules in the brain to enable the estimation of the brain's anatomical fiber tracts at a relatively high resolution. In particular, tractographic methods can be used to generate whole-brain anatomical connectivity matrix where each element provides an estimate of the connectivity strength between the correspo… ▽ More Diffusion Magnetic Resonance Imaging (MRI) exploits the anisotropic diffusion of water molecules in the brain to enable the estimation of the brain's anatomical fiber tracts at a relatively high resolution. In particular, tractographic methods can be used to generate whole-brain anatomical connectivity matrix where each element provides an estimate of the connectivity strength between the corresponding voxels. Structural brain networks are built using the connectivity information and a predefined brain parcellation, where the nodes of the network represent the brain regions and the edge weights capture the connectivity strengths between the corresponding brain regions. This paper introduces a number of novel scalable methods to generate and analyze structural brain networks with a varying number of nodes. In particular, we introduce a new parallel algorithm to quickly generate large scale connectivity-based parcellations for which voxels in a region possess highly similar connectivity patterns to the rest of the regions. We show that the corresponding regional structural consistency is always superior to randomly generated parcellations over a wide range of parcellation sizes. Corresponding brain networks with a varying number of nodes are analyzed using standard graph-theorectic measures, as well as, new measures derived from spectral graph theory. Our results indicate increasingly more statistical power of brain networks with larger numbers of nodes and the relatively unique shape of the spectral profile of large brain networks relative to other well-known networks. △ Less

Submitted 13 September, 2016; originally announced September 2016.