-
MUGC: Machine Generated versus User Generated Content Detection
Authors:
Yaqi Xie,
Anjali Rawal,
Yu**g Cen,
Dixuan Zhao,
Sunil K Narang,
Shanu Sushmita
Abstract:
As advanced modern systems like deep neural networks (DNNs) and generative AI continue to enhance their capabilities in producing convincing and realistic content, the need to distinguish between user-generated and machine generated content is becoming increasingly evident. In this research, we undertake a comparative evaluation of eight traditional machine-learning algorithms to distinguish betwe…
▽ More
As advanced modern systems like deep neural networks (DNNs) and generative AI continue to enhance their capabilities in producing convincing and realistic content, the need to distinguish between user-generated and machine generated content is becoming increasingly evident. In this research, we undertake a comparative evaluation of eight traditional machine-learning algorithms to distinguish between machine-generated and human-generated data across three diverse datasets: Poems, Abstracts, and Essays. Our results indicate that traditional methods demonstrate a high level of accuracy in identifying machine-generated data, reflecting the documented effectiveness of popular pre-trained models like RoBERT. We note that machine-generated texts tend to be shorter and exhibit less word variety compared to human-generated content. While specific domain-related keywords commonly utilized by humans, albeit disregarded by current LLMs (Large Language Models), may contribute to this high detection accuracy, we show that deeper word representations like word2vec can capture subtle semantic variances. Furthermore, readability, bias, moral, and affect comparisons reveal a discernible contrast between machine-generated and human generated content. There are variations in expression styles and potentially underlying biases in the data sources (human and machine-generated). This study provides valuable insights into the advancing capacities and challenges associated with machine-generated content across various domains.
△ Less
Submitted 28 March, 2024;
originally announced March 2024.
-
On the Equivalence of Graph Convolution and Mixup
Authors:
Xiaotian Han,
Hanqing Zeng,
Yu Chen,
Shaoliang Nie,
**gzhou Liu,
Kanika Narang,
Zahra Shakeri,
Karthik Abinav Sankararaman,
Song Jiang,
Madian Khabsa,
Qifan Wang,
Xia Hu
Abstract:
This paper investigates the relationship between graph convolution and Mixup techniques. Graph convolution in a graph neural network involves aggregating features from neighboring samples to learn representative features for a specific node or sample. On the other hand, Mixup is a data augmentation technique that generates new examples by averaging features and one-hot labels from multiple samples…
▽ More
This paper investigates the relationship between graph convolution and Mixup techniques. Graph convolution in a graph neural network involves aggregating features from neighboring samples to learn representative features for a specific node or sample. On the other hand, Mixup is a data augmentation technique that generates new examples by averaging features and one-hot labels from multiple samples. One commonality between these techniques is their utilization of information from multiple samples to derive feature representation. This study aims to explore whether a connection exists between these two approaches. Our investigation reveals that, under two mild conditions, graph convolution can be viewed as a specialized form of Mixup that is applied during both the training and testing phases. The two conditions are: 1) \textit{Homophily Relabel} - assigning the target node's label to all its neighbors, and 2) \textit{Test-Time Mixup} - Mixup the feature during the test time. We establish this equivalence mathematically by demonstrating that graph convolution networks (GCN) and simplified graph convolution (SGC) can be expressed as a form of Mixup. We also empirically verify the equivalence by training an MLP using the two conditions to achieve comparable performance.
△ Less
Submitted 29 September, 2023;
originally announced October 2023.
-
Meta-training with Demonstration Retrieval for Efficient Few-shot Learning
Authors:
Aaron Mueller,
Kanika Narang,
Lambert Mathias,
Qifan Wang,
Hamed Firooz
Abstract:
Large language models show impressive results on few-shot NLP tasks. However, these models are memory and computation-intensive. Meta-training allows one to leverage smaller models for few-shot generalization in a domain-general and task-agnostic manner; however, these methods alone results in models that may not have sufficient parameterization or knowledge to adapt quickly to a large variety of…
▽ More
Large language models show impressive results on few-shot NLP tasks. However, these models are memory and computation-intensive. Meta-training allows one to leverage smaller models for few-shot generalization in a domain-general and task-agnostic manner; however, these methods alone results in models that may not have sufficient parameterization or knowledge to adapt quickly to a large variety of tasks. To overcome this issue, we propose meta-training with demonstration retrieval, where we use a dense passage retriever to retrieve semantically similar labeled demonstrations to each example for more varied supervision. By separating external knowledge from model parameters, we can use meta-training to train parameter-efficient models that generalize well on a larger variety of tasks. We construct a meta-training set from UnifiedQA and CrossFit, and propose a demonstration bank based on UnifiedQA tasks. To our knowledge, our work is the first to combine retrieval with meta-training, to use DPR models to retrieve demonstrations, and to leverage demonstrations from many tasks simultaneously, rather than randomly sampling demonstrations from the training set of the target task. Our approach outperforms a variety of targeted parameter-efficient and retrieval-augmented few-shot methods on QA, NLI, and text classification tasks (including SQuAD, QNLI, and TREC). Our approach can be meta-trained and fine-tuned quickly on a single GPU.
△ Less
Submitted 30 June, 2023;
originally announced July 2023.
-
Measuring Self-Supervised Representation Quality for Downstream Classification using Discriminative Features
Authors:
Neha Kalibhat,
Kanika Narang,
Hamed Firooz,
Maziar Sanjabi,
Soheil Feizi
Abstract:
Self-supervised learning (SSL) has shown impressive results in downstream classification tasks. However, there is limited work in understanding their failure modes and interpreting their learned representations. In this paper, we study the representation space of state-of-the-art self-supervised models including SimCLR, SwaV, MoCo, BYOL, DINO, SimSiam, VICReg and Barlow Twins. Without the use of c…
▽ More
Self-supervised learning (SSL) has shown impressive results in downstream classification tasks. However, there is limited work in understanding their failure modes and interpreting their learned representations. In this paper, we study the representation space of state-of-the-art self-supervised models including SimCLR, SwaV, MoCo, BYOL, DINO, SimSiam, VICReg and Barlow Twins. Without the use of class label information, we discover discriminative features that correspond to unique physical attributes in images, present mostly in correctly-classified representations. Using these features, we can compress the representation space by up to 40% without significantly affecting linear classification performance. We then propose Self-Supervised Representation Quality Score (or Q-Score), an unsupervised score that can reliably predict if a given sample is likely to be mis-classified during linear evaluation, achieving AUPRC of 91.45 on ImageNet-100 and 78.78 on ImageNet-1K. Q-Score can also be used as a regularization term on pre-trained encoders to remedy low-quality representations. Fine-tuning with Q-Score regularization can boost the linear probing accuracy of SSL models by up to 5.8% on ImageNet-100 and 3.7% on ImageNet-1K compared to their baselines. Finally, using gradient heatmaps and Salient ImageNet masks, we define a metric to quantify the interpretability of each representation. We show that discriminative features are strongly correlated to core attributes and, enhancing these features through Q-score regularization makes SSL representations more interpretable.
△ Less
Submitted 12 December, 2023; v1 submitted 3 March, 2022;
originally announced March 2022.
-
An Induced Multi-Relational Framework for Answer Selection in Community Question Answer Platforms
Authors:
Kanika Narang,
Chaoqi Yang,
Adit Krishnan,
Junting Wang,
Hari Sundaram,
Carolyn Sutter
Abstract:
This paper addresses the question of identifying the best candidate answer to a question on Community Question Answer (CQA) forums. The problem is important because Individuals often visit CQA forums to seek answers to nuanced questions. We develop a novel induced relational graph convolutional network (IR-GCN) framework to address the question. We make three contributions. First, we introduce a m…
▽ More
This paper addresses the question of identifying the best candidate answer to a question on Community Question Answer (CQA) forums. The problem is important because Individuals often visit CQA forums to seek answers to nuanced questions. We develop a novel induced relational graph convolutional network (IR-GCN) framework to address the question. We make three contributions. First, we introduce a modular framework that separates the construction of the graph with the label selection mechanism. We use equivalence relations to induce a graph comprising cliques and identify two label assignment mechanisms---label contrast, label sharing. Then, we show how to encode these assignment mechanisms in GCNs. Second, we show that encoding contrast creates discriminative magnification---enhancing the separation between nodes in the embedding space. Third, we show a surprising result---boosting techniques improve learning over familiar stacking, fusion, or aggregation approaches for neural architectures. We show strong results over the state-of-the-art neural baselines in extensive experiments on 50 StackExchange communities.
△ Less
Submitted 16 November, 2019;
originally announced November 2019.
-
Discovering Archetypes to Interpret Evolution of Individual Behavior
Authors:
Kanika Narang,
Austin Chung,
Hari Sundaram,
Snigdha Chaturvedi
Abstract:
In this paper, we aim to discover archetypical patterns of individual evolution in large social networks. In our work, an archetype comprises of $\textit{progressive stages}$ of distinct behavior. We introduce a novel Gaussian Hidden Markov Model (G-HMM) Cluster to identify archetypes of evolutionary patterns. G-HMMs allow for: near limitless behavioral variation; imposing constraints on how indiv…
▽ More
In this paper, we aim to discover archetypical patterns of individual evolution in large social networks. In our work, an archetype comprises of $\textit{progressive stages}$ of distinct behavior. We introduce a novel Gaussian Hidden Markov Model (G-HMM) Cluster to identify archetypes of evolutionary patterns. G-HMMs allow for: near limitless behavioral variation; imposing constraints on how individuals can evolve; different evolutionary rates; and are parsimonious. Our experiments with Academic and StackExchange dataset discover insightful archetypes. We identify four archetypes for researchers: $\textit{Steady}$, $\textit{Diverse, Evolving and Diffuse}$. We observe clear differences in the evolution of male and female researchers within the same archetype. Specifically, women and men differ within an archetype (e.g. Diverse) in how they start, how they transition and the time spent in mid-career. We also found that the differences in grant income are better explained by the differences in archetype than by differences in gender. For StackOverflow, discovered archetypes could be labeled as $\textit{Experts, Seekers, Enthusiasts, and Facilitators}$. We have strong quantitative results with competing baselines for activity prediction and perplexity. For future session prediction, the proposed G-HMM cluster model improves by an average of $32\%$ for different Stack Exchanges and $24\%$ for Academic dataset. Our model also exhibits lower perplexity than the baselines.
△ Less
Submitted 4 April, 2019; v1 submitted 14 February, 2019;
originally announced February 2019.
-
Localized Iterative Methods for Interpolation in Graph Structured Data
Authors:
Sunil K. Narang,
Akshay Gadde,
Eduard Sanou,
Antonio Ortega
Abstract:
In this paper, we present two localized graph filtering based methods for interpolating graph signals defined on the vertices of arbitrary graphs from only a partial set of samples. The first method is an extension of previous work on reconstructing bandlimited graph signals from partially observed samples. The iterative graph filtering approach very closely approximates the solution proposed in t…
▽ More
In this paper, we present two localized graph filtering based methods for interpolating graph signals defined on the vertices of arbitrary graphs from only a partial set of samples. The first method is an extension of previous work on reconstructing bandlimited graph signals from partially observed samples. The iterative graph filtering approach very closely approximates the solution proposed in the that work, while being computationally more efficient. As an alternative, we propose a regularization based framework in which we define the cost of reconstruction to be a combination of smoothness of the graph signal and the reconstruction error with respect to the known samples, and find solutions that minimize this cost. We provide both a closed form solution and a computationally efficient iterative solution of the optimization problem. The experimental results on the recommendation system datasets demonstrate effectiveness of the proposed methods.
△ Less
Submitted 9 October, 2013;
originally announced October 2013.
-
Bilateral Filter: Graph Spectral Interpretation and Extensions
Authors:
Akshay Gadde,
Sunil K Narang,
Antonio Ortega
Abstract:
In this paper we study the bilateral filter proposed by Tomasi and Manduchi, as a spectral domain transform defined on a weighted graph. The nodes of this graph represent the pixels in the image and a graph signal defined on the nodes represents the intensity values. Edge weights in the graph correspond to the bilateral filter coefficients and hence are data adaptive. Spectrum of a graph is define…
▽ More
In this paper we study the bilateral filter proposed by Tomasi and Manduchi, as a spectral domain transform defined on a weighted graph. The nodes of this graph represent the pixels in the image and a graph signal defined on the nodes represents the intensity values. Edge weights in the graph correspond to the bilateral filter coefficients and hence are data adaptive. Spectrum of a graph is defined in terms of the eigenvalues and eigenvectors of the graph Laplacian matrix. We use this spectral interpretation to generalize the bilateral filter and propose more flexible and application specific spectral designs of bilateral-like filters. We show that these spectral filters can be implemented with k-iterative bilateral filtering operations and do not require expensive diagonalization of the Laplacian matrix.
△ Less
Submitted 11 March, 2013;
originally announced March 2013.
-
The Emerging Field of Signal Processing on Graphs: Extending High-Dimensional Data Analysis to Networks and Other Irregular Domains
Authors:
David I Shuman,
Sunil K. Narang,
Pascal Frossard,
Antonio Ortega,
Pierre Vandergheynst
Abstract:
In applications such as social, energy, transportation, sensor, and neuronal networks, high-dimensional data naturally reside on the vertices of weighted graphs. The emerging field of signal processing on graphs merges algebraic and spectral graph theoretic concepts with computational harmonic analysis to process such signals on graphs. In this tutorial overview, we outline the main challenges of…
▽ More
In applications such as social, energy, transportation, sensor, and neuronal networks, high-dimensional data naturally reside on the vertices of weighted graphs. The emerging field of signal processing on graphs merges algebraic and spectral graph theoretic concepts with computational harmonic analysis to process such signals on graphs. In this tutorial overview, we outline the main challenges of the area, discuss different ways to define graph spectral domains, which are the analogues to the classical frequency domain, and highlight the importance of incorporating the irregular structures of graph data domains when processing signals on graphs. We then review methods to generalize fundamental operations such as filtering, translation, modulation, dilation, and downsampling to the graph setting, and survey the localized, multiscale transforms that have been proposed to efficiently extract information from high-dimensional data on graphs. We conclude with a brief discussion of open issues and possible extensions.
△ Less
Submitted 10 March, 2013; v1 submitted 31 October, 2012;
originally announced November 2012.
-
Compact Support Biorthogonal Wavelet Filterbanks for Arbitrary Undirected Graphs
Authors:
Sunil K. Narang,
Antonio Ortega
Abstract:
In our recent work, we proposed the design of perfect reconstruction orthogonal wavelet filterbanks, called graph- QMF, for arbitrary undirected weighted graphs. In that formulation we first designed "one-dimensional" two-channel filterbanks on bipartite graphs, and then extended them to "multi-dimensional" separable two-channel filterbanks for arbitrary graphs via a bipartite subgraph decompositi…
▽ More
In our recent work, we proposed the design of perfect reconstruction orthogonal wavelet filterbanks, called graph- QMF, for arbitrary undirected weighted graphs. In that formulation we first designed "one-dimensional" two-channel filterbanks on bipartite graphs, and then extended them to "multi-dimensional" separable two-channel filterbanks for arbitrary graphs via a bipartite subgraph decomposition. We specifically designed wavelet filters based on the spectral decomposition of the graph, and stated necessary and sufficient conditions for a two-channel graph filter-bank on bipartite graphs to provide aliasing-cancellation, perfect reconstruction and orthogonal set of basis (orthogonality). While, the exact graph-QMF designs satisfy all the above conditions, they are not exactly k-hop localized on the graph. In this paper, we relax the condition of orthogonality to design a biorthogonal pair of graph-wavelets that can have compact spatial spread and still satisfy the perfect reconstruction conditions. The design is analogous to the standard Cohen-Daubechies-Feauveau's (CDF) construction of factorizing a maximally-flat Daubechies half-band filter. Preliminary results demonstrate that the proposed filterbanks can be useful for both standard signal processing applications as well as for signals defined on arbitrary graphs.
Note: Code examples from this paper are available at http://biron.usc.edu/wiki/index.php/Graph Filterbanks
△ Less
Submitted 19 November, 2012; v1 submitted 30 October, 2012;
originally announced October 2012.
-
Perfect Reconstruction Two-Channel Wavelet Filter-Banks for Graph Structured Data
Authors:
Sunil K. Narang,
Antonio Ortega
Abstract:
In this work we propose the construction of two-channel wavelet filterbanks for analyzing functions defined on the vertices of any arbitrary finite weighted undirected graph. These graph based functions are referred to as graph-signals as we build a framework in which many concepts from the classical signal processing domain, such as Fourier decomposition, signal filtering and downsampling can be…
▽ More
In this work we propose the construction of two-channel wavelet filterbanks for analyzing functions defined on the vertices of any arbitrary finite weighted undirected graph. These graph based functions are referred to as graph-signals as we build a framework in which many concepts from the classical signal processing domain, such as Fourier decomposition, signal filtering and downsampling can be extended to graph domain. Especially, we observe a spectral folding phenomenon in bipartite graphs which occurs during downsampling of these graphs and produces aliasing in graph signals. This property of bipartite graphs, allows us to design critically sampled two-channel filterbanks, and we propose quadrature mirror filters (referred to as graph-QMF) for bipartite graph which cancel aliasing and lead to perfect reconstruction. For arbitrary graphs we present a bipartite subgraph decomposition which produces an edge-disjoint collection of bipartite subgraphs. Graph-QMFs are then constructed on each bipartite subgraph leading to "multi-dimensional" separable wavelet filterbanks on graphs. Our proposed filterbanks are critically sampled and we state necessary and sufficient conditions for orthogonality, aliasing cancellation and perfect reconstruction. The filterbanks are realized by Chebychev polynomial approximations.
△ Less
Submitted 1 December, 2011; v1 submitted 18 June, 2011;
originally announced June 2011.
