Showing 1–26 of 26 results for author: Parekh, A

Learning To See But Forgetting To Follow: Visual Instruction Tuning Makes LLMs More Prone To Jailbreak Attacks

Authors: Georgios Pantazopoulos, Amit Parekh, Malvina Nikandrou, Alessandro Suglia

Abstract: Augmenting Large Language Models (LLMs) with image-understanding capabilities has resulted in a boom of high-performing Vision-Language models (VLMs). While studying the alignment of LLMs to human values has received widespread attention, the safety of VLMs has not received the same attention. In this paper, we explore the impact of jailbreaking on three state-of-the-art VLMs, each using a distinc… ▽ More Augmenting Large Language Models (LLMs) with image-understanding capabilities has resulted in a boom of high-performing Vision-Language models (VLMs). While studying the alignment of LLMs to human values has received widespread attention, the safety of VLMs has not received the same attention. In this paper, we explore the impact of jailbreaking on three state-of-the-art VLMs, each using a distinct modeling approach. By comparing each VLM to their respective LLM backbone, we find that each VLM is more susceptible to jailbreaking. We consider this as an undesirable outcome from visual instruction-tuning, which imposes a forgetting effect on an LLM's safety guardrails. Therefore, we provide recommendations for future work based on evaluation strategies that aim to highlight the weaknesses of a VLM, as well as take safety measures into account during visual instruction tuning. △ Less

Submitted 7 May, 2024; originally announced May 2024.

Noise in Relation Classification Dataset TACRED: Characterization and Reduction

Authors: Akshay Parekh, Ashish Anand, Amit Awekar

Abstract: The overarching objective of this paper is two-fold. First, to explore model-based approaches to characterize the primary cause of the noise. in the RE dataset TACRED Second, to identify the potentially noisy instances. Towards the first objective, we analyze predictions and performance of state-of-the-art (SOTA) models to identify the root cause of noise in the dataset. Our analysis of TACRED sho… ▽ More The overarching objective of this paper is two-fold. First, to explore model-based approaches to characterize the primary cause of the noise. in the RE dataset TACRED Second, to identify the potentially noisy instances. Towards the first objective, we analyze predictions and performance of state-of-the-art (SOTA) models to identify the root cause of noise in the dataset. Our analysis of TACRED shows that the majority of the noise in the dataset originates from the instances labeled as no-relation which are negative examples. For the second objective, we explore two nearest-neighbor-based strategies to automatically identify potentially noisy examples for elimination and reannotation. Our first strategy, referred to as Intrinsic Strategy (IS), is based on the assumption that positive examples are clean. Thus, we have used false-negative predictions to identify noisy negative examples. Whereas, our second approach, referred to as Extrinsic Strategy, is based on using a clean subset of the dataset to identify potentially noisy negative examples. Finally, we retrained the SOTA models on the eliminated and reannotated dataset. Our empirical results based on two SOTA models trained on TACRED-E following the IS show an average 4% F1-score improvement, whereas reannotation (TACRED-R) does not improve the original results. However, following ES, SOTA models show the average F1-score improvement of 3.8% and 4.4% when trained on respective eliminated (TACRED-EN) and reannotated (TACRED-RN) datasets respectively. We further extended the ES for cleaning positive examples as well, which resulted in an average performance improvement of 5.8% and 5.6% for the eliminated (TACRED-ENP) and reannotated (TACRED-RNP) datasets respectively. △ Less

Submitted 20 November, 2023; originally announced November 2023.

Comments: Work in Progress

Multitask Multimodal Prompted Training for Interactive Embodied Task Completion

Authors: Georgios Pantazopoulos, Malvina Nikandrou, Amit Parekh, Bhathiya Hemanthage, Arash Eshghi, Ioannis Konstas, Verena Rieser, Oliver Lemon, Alessandro Suglia

Abstract: Interactive and embodied tasks pose at least two fundamental challenges to existing Vision & Language (VL) models, including 1) grounding language in trajectories of actions and observations, and 2) referential disambiguation. To tackle these challenges, we propose an Embodied MultiModal Agent (EMMA): a unified encoder-decoder model that reasons over images and trajectories, and casts action predi… ▽ More Interactive and embodied tasks pose at least two fundamental challenges to existing Vision & Language (VL) models, including 1) grounding language in trajectories of actions and observations, and 2) referential disambiguation. To tackle these challenges, we propose an Embodied MultiModal Agent (EMMA): a unified encoder-decoder model that reasons over images and trajectories, and casts action prediction as multimodal text generation. By unifying all tasks as text generation, EMMA learns a language of actions which facilitates transfer across tasks. Different to previous modular approaches with independently trained components, we use a single multitask model where each task contributes to goal completion. EMMA performs on par with similar models on several VL benchmarks and sets a new state-of-the-art performance (36.81% success rate) on the Dialog-guided Task Completion (DTC), a benchmark to evaluate dialog-guided agents in the Alexa Arena △ Less

Submitted 7 November, 2023; originally announced November 2023.

Comments: EMNLP 2023

iLab at SemEval-2023 Task 11 Le-Wi-Di: Modelling Disagreement or Modelling Perspectives?

Authors: Nikolas Vitsakis, Amit Parekh, Tanvi Dinkar, Gavin Abercrombie, Ioannis Konstas, Verena Rieser

Abstract: There are two competing approaches for modelling annotator disagreement: distributional soft-labelling approaches (which aim to capture the level of disagreement) or modelling perspectives of individual annotators or groups thereof. We adapt a multi-task architecture -- which has previously shown success in modelling perspectives -- to evaluate its performance on the SEMEVAL Task 11. We do so by c… ▽ More There are two competing approaches for modelling annotator disagreement: distributional soft-labelling approaches (which aim to capture the level of disagreement) or modelling perspectives of individual annotators or groups thereof. We adapt a multi-task architecture -- which has previously shown success in modelling perspectives -- to evaluate its performance on the SEMEVAL Task 11. We do so by combining both approaches, i.e. predicting individual annotator perspectives as an interim step towards predicting annotator disagreement. Despite its previous success, we found that a multi-task approach performed poorly on datasets which contained distinct annotator opinions, suggesting that this approach may not always be suitable when modelling perspectives. Furthermore, our results explain that while strongly perspectivist approaches might not achieve state-of-the-art performance according to evaluation metrics used by distributional approaches, our approach allows for a more nuanced understanding of individual perspectives present in the data. We argue that perspectivist approaches are preferable because they enable decision makers to amplify minority views, and that it is important to re-evaluate metrics to reflect this goal. △ Less

Submitted 10 May, 2023; originally announced May 2023.

Comments: To appear in the Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). Association for Computational Linguistics, 2023

Transmit Power Control for Indoor Small Cells: A Method Based on Federated Reinforcement Learning

Authors: Peizheng Li, Hakan Erdol, Keith Briggs, Xiaoyang Wang, Robert Piechocki, Abdelrahim Ahmad, Rui Inacio, Shipra Kapoor, Angela Doufexi, Arjun Parekh

Abstract: Setting the transmit power setting of 5G cells has been a long-term topic of discussion, as optimized power settings can help reduce interference and improve the quality of service to users. Recently, machine learning (ML)-based, especially reinforcement learning (RL)-based control methods have received much attention. However, there is little discussion about the generalisation ability of the tra… ▽ More Setting the transmit power setting of 5G cells has been a long-term topic of discussion, as optimized power settings can help reduce interference and improve the quality of service to users. Recently, machine learning (ML)-based, especially reinforcement learning (RL)-based control methods have received much attention. However, there is little discussion about the generalisation ability of the trained RL models. This paper points out that an RL agent trained in a specific indoor environment is room-dependent, and cannot directly serve new heterogeneous environments. Therefore, in the context of Open Radio Access Network (O-RAN), this paper proposes a distributed cell power-control scheme based on Federated Reinforcement Learning (FRL). Models in different indoor environments are aggregated to the global model during the training process, and then the central server broadcasts the updated model back to each client. The model will also be used as the base model for adaptive training in the new environment. The simulation results show that the FRL model has similar performance to a single RL agent, and both are better than the random power allocation method and exhaustive search method. The results of the generalisation test show that using the FRL model as the base model improves the convergence speed of the model in the new environment. △ Less

Submitted 31 August, 2022; originally announced September 2022.

Comments: 7 pages, 5 figures. This paper has been accepted by 2022 IEEE 96th Vehicular Technology Conference (VTC2022-Fall)

Variational Autoencoder Assisted Neural Network Likelihood RSRP Prediction Model

Authors: Peizheng Li, Xiaoyang Wang, Robert Piechocki, Shipra Kapoor, Angela Doufexi, Arjun Parekh

Abstract: Measuring customer experience on mobile data is of utmost importance for global mobile operators. The reference signal received power (RSRP) is one of the important indicators for current mobile network management, evaluation and monitoring. Radio data gathered through the minimization of drive test (MDT), a 3GPP standard technique, is commonly used for radio network analysis. Collecting MDT data… ▽ More Measuring customer experience on mobile data is of utmost importance for global mobile operators. The reference signal received power (RSRP) is one of the important indicators for current mobile network management, evaluation and monitoring. Radio data gathered through the minimization of drive test (MDT), a 3GPP standard technique, is commonly used for radio network analysis. Collecting MDT data in different geographical areas is inefficient and constrained by the terrain conditions and user presence, hence is not an adequate technique for dynamic radio environments. In this paper, we study a generative model for RSRP prediction, exploiting MDT data and a digital twin (DT), and propose a data-driven, two-tier neural network (NN) model. In the first tier, environmental information related to user equipment (UE), base stations (BS) and network key performance indicators (KPI) are extracted through a variational autoencoder (VAE). The second tier is designed as a likelihood model. Here, the environmental features and real MDT data features are adopted, formulating an integrated training process. On validation, our proposed model that uses real-world data demonstrates an accuracy improvement of about 20% or more compared with the empirical model and about 10% when compared with a fully connected prediction network. △ Less

Submitted 27 June, 2022; originally announced July 2022.

Comments: 6 pages, 4 figures. This paper has been accepted for publication in PIMRC 2022

Sim2real for Reinforcement Learning Driven Next Generation Networks

Authors: Peizheng Li, Jonathan Thomas, Xiaoyang Wang, Hakan Erdol, Abdelrahim Ahmad, Rui Inacio, Shipra Kapoor, Arjun Parekh, Angela Doufexi, Arman Shojaeifard, Robert Piechocki

Abstract: The next generation of networks will actively embrace artificial intelligence (AI) and machine learning (ML) technologies for automation networks and optimal network operation strategies. The emerging network structure represented by Open RAN (O-RAN) conforms to this trend, and the radio intelligent controller (RIC) at the centre of its specification serves as an ML applications host. Various ML m… ▽ More The next generation of networks will actively embrace artificial intelligence (AI) and machine learning (ML) technologies for automation networks and optimal network operation strategies. The emerging network structure represented by Open RAN (O-RAN) conforms to this trend, and the radio intelligent controller (RIC) at the centre of its specification serves as an ML applications host. Various ML models, especially Reinforcement Learning (RL) models, are regarded as the key to solving RAN-related multi-objective optimization problems. However, it should be recognized that most of the current RL successes are confined to abstract and simplified simulation environments, which may not directly translate to high performance in complex real environments. One of the main reasons is the modelling gap between the simulation and the real environment, which could make the RL agent trained by simulation ill-equipped for the real environment. This issue is termed as the sim2real gap. This article brings to the fore the sim2real challenge within the context of O-RAN. Specifically, it emphasizes the characteristics, and benefits that the digital twins (DT) could have as a place for model development and verification. Several use cases are presented to exemplify and demonstrate failure modes of the simulations trained RL model in real environments. The effectiveness of DT in assisting the development of RL algorithms is discussed. Then the current state of the art learning-based methods commonly used to overcome the sim2real challenge are presented. Finally, the development and deployment concerns for the RL applications realisation in O-RAN are discussed from the view of the potential issues like data interaction, environment bottlenecks, and algorithm design. △ Less

Submitted 8 June, 2022; originally announced June 2022.

Comments: 7 pages, 4 figures

Towards Understanding Gender-Seniority Compound Bias in Natural Language Generation

Authors: Samhita Honnavalli, Aesha Parekh, Lily Ou, Sophie Groenwold, Sharon Levy, Vicente Ordonez, William Yang Wang

Abstract: Women are often perceived as junior to their male counterparts, even within the same job titles. While there has been significant progress in the evaluation of gender bias in natural language processing (NLP), existing studies seldom investigate how biases toward gender groups change when compounded with other societal biases. In this work, we investigate how seniority impacts the degree of gender… ▽ More Women are often perceived as junior to their male counterparts, even within the same job titles. While there has been significant progress in the evaluation of gender bias in natural language processing (NLP), existing studies seldom investigate how biases toward gender groups change when compounded with other societal biases. In this work, we investigate how seniority impacts the degree of gender bias exhibited in pretrained neural generation models by introducing a novel framework for probing compound bias. We contribute a benchmark robustness-testing dataset spanning two domains, U.S. senatorship and professorship, created using a distant-supervision method. Our dataset includes human-written text with underlying ground truth and paired counterfactuals. We then examine GPT-2 perplexity and the frequency of gendered language in generated text. Our results show that GPT-2 amplifies bias by considering women as junior and men as senior more often than the ground truth in both domains. These results suggest that NLP applications built using GPT-2 may harm women in professional capacities. △ Less

Submitted 19 May, 2022; originally announced May 2022.

Comments: 6 pages, LREC 2022

Budget Sensitive Reannotation of Noisy Relation Classification Data Using Label Hierarchy

Authors: Akshay Parekh, Ashish Anand, Amit Awekar

Abstract: Large crowd-sourced datasets are often noisy and relation classification (RC) datasets are no exception. Reannotating the entire dataset is one probable solution however it is not always viable due to time and budget constraints. This paper addresses the problem of efficient reannotation of a large noisy dataset for the RC. Our goal is to catch more annotation errors in the dataset while reannotat… ▽ More Large crowd-sourced datasets are often noisy and relation classification (RC) datasets are no exception. Reannotating the entire dataset is one probable solution however it is not always viable due to time and budget constraints. This paper addresses the problem of efficient reannotation of a large noisy dataset for the RC. Our goal is to catch more annotation errors in the dataset while reannotating fewer instances. Existing work on RC dataset reannotation lacks the flexibility about how much data to reannotate. We introduce the concept of a reannotation budget to overcome this limitation. The immediate follow-up problem is: Given a specific reannotation budget, which subset of the data should we reannotate? To address this problem, we present two strategies to selectively reannotate RC datasets. Our strategies utilize the taxonomic hierarchy of relation labels. The intuition of our work is to rely on the graph distance between actual and predicted relation labels in the label hierarchy graph. We evaluate our reannotation strategies on the well-known TACRED dataset. We design our experiments to answer three specific research questions. First, does our strategy select novel candidates for reannotation? Second, for a given reannotation budget is our reannotation strategy more efficient at catching annotation errors? Third, what is the impact of data reannotation on RC model performance measurement? Experimental results show that our both reannotation strategies are novel and efficient. Our analysis indicates that the current reported performance of RC models on noisy TACRED data is inflated. △ Less

Submitted 26 December, 2021; originally announced December 2021.

RLOps: Development Life-cycle of Reinforcement Learning Aided Open RAN

Authors: Peizheng Li, Jonathan Thomas, Xiaoyang Wang, Ahmed Khalil, Abdelrahim Ahmad, Rui Inacio, Shipra Kapoor, Arjun Parekh, Angela Doufexi, Arman Shojaeifard, Robert Piechocki

Abstract: Radio access network (RAN) technologies continue to evolve, with Open RAN gaining the most recent momentum. In the O-RAN specifications, the RAN intelligent controllers (RICs) are software-defined orchestration and automation functions for the intelligent management of RAN. This article introduces principles for machine learning (ML), in particular, reinforcement learning (RL) applications in the… ▽ More Radio access network (RAN) technologies continue to evolve, with Open RAN gaining the most recent momentum. In the O-RAN specifications, the RAN intelligent controllers (RICs) are software-defined orchestration and automation functions for the intelligent management of RAN. This article introduces principles for machine learning (ML), in particular, reinforcement learning (RL) applications in the O-RAN stack. Furthermore, we review the state-of-the-art research in wireless networks and cast it onto the RAN framework and the hierarchy of the O-RAN architecture. We provide a taxonomy for the challenges faced by ML/RL models throughout the development life-cycle: from the system specification to production deployment (data acquisition, model design, testing and management, etc.). To address the challenges, we integrate a set of existing MLOps principles with unique characteristics when RL agents are considered. This paper discusses a systematic model development, testing and validation life-cycle, termed: RLOps. We discuss fundamental parts of RLOps, which include: model specification, development, production environment serving, operations monitoring and safety/security. Based on these principles, we propose the best practices for RLOps to achieve an automated and reproducible model development process. At last, a holistic data analytics platform rooted in the O-RAN deployment is designed and implemented, aiming to embrace and fulfil the aforementioned principles and best practices of RLOps. △ Less

Submitted 25 November, 2022; v1 submitted 12 November, 2021; originally announced November 2021.

Comments: 17 pages, 6 figrues

Journal ref: IEEE Access (2022), vol. 10, pp. 113808-113826

Document Structure aware Relational Graph Convolutional Networks for Ontology Population

Authors: Abhay M Shalghar, Ayush Kumar, Balaji Ganesan, Aswin Kannan, Akshay Parekh, Shobha G

Abstract: Ontologies comprising of concepts, their attributes, and relationships are used in many knowledge based AI systems. While there have been efforts towards populating domain specific ontologies, we examine the role of document structure in learning ontological relationships between concepts in any document corpus. Inspired by ideas from hypernym discovery and explainability, our method performs abou… ▽ More Ontologies comprising of concepts, their attributes, and relationships are used in many knowledge based AI systems. While there have been efforts towards populating domain specific ontologies, we examine the role of document structure in learning ontological relationships between concepts in any document corpus. Inspired by ideas from hypernym discovery and explainability, our method performs about 15 points more accurate than a stand-alone R-GCN model for this task. △ Less

Submitted 12 April, 2022; v1 submitted 26 April, 2021; originally announced April 2021.

Comments: 8 pages single column, 5 figures. DLG4NLP Workshop at ICLR 2022

Self-play Learning Strategies for Resource Assignment in Open-RAN Networks

Authors: Xiaoyang Wang, Jonathan D Thomas, Robert J Piechocki, Shipra Kapoor, Raul Santos-Rodriguez, Arjun Parekh

Abstract: Open Radio Access Network (ORAN) is being developed with an aim to democratise access and lower the cost of future mobile data networks, supporting network services with various QoS requirements, such as massive IoT and URLLC. In ORAN, network functionality is dis-aggregated into remote units (RUs), distributed units (DUs) and central units (CUs), which allows flexible software on Commercial-Off-T… ▽ More Open Radio Access Network (ORAN) is being developed with an aim to democratise access and lower the cost of future mobile data networks, supporting network services with various QoS requirements, such as massive IoT and URLLC. In ORAN, network functionality is dis-aggregated into remote units (RUs), distributed units (DUs) and central units (CUs), which allows flexible software on Commercial-Off-The-Shelf (COTS) deployments. Furthermore, the map** of variable RU requirements to local mobile edge computing centres for future centralized processing would significantly reduce the power consumption in cellular networks. In this paper, we study the RU-DU resource assignment problem in an ORAN system, modelled as a 2D bin packing problem. A deep reinforcement learning-based self-play approach is proposed to achieve efficient RU-DU resource management, with AlphaGo Zero inspired neural Monte-Carlo Tree Search (MCTS). Experiments on representative 2D bin packing environment and real sites data show that the self-play learning strategy achieves intelligent RU-DU resource assignment for different network conditions. △ Less

Submitted 3 March, 2021; originally announced March 2021.

MSC Class: 93-10 ACM Class: C.2.3; I.2.8

Investigating African-American Vernacular English in Transformer-Based Text Generation

Authors: Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, William Yang Wang

Abstract: The growth of social media has encouraged the written use of African American Vernacular English (AAVE), which has traditionally been used only in oral contexts. However, NLP models have historically been developed using dominant English varieties, such as Standard American English (SAE), due to text corpora availability. We investigate the performance of GPT-2 on AAVE text by creating a dataset o… ▽ More The growth of social media has encouraged the written use of African American Vernacular English (AAVE), which has traditionally been used only in oral contexts. However, NLP models have historically been developed using dominant English varieties, such as Standard American English (SAE), due to text corpora availability. We investigate the performance of GPT-2 on AAVE text by creating a dataset of intent-equivalent parallel AAVE/SAE tweet pairs, thereby isolating syntactic structure and AAVE- or SAE-specific language for each pair. We evaluate each sample and its GPT-2 generated text with pretrained sentiment classifiers and find that while AAVE text results in more classifications of negative sentiment than SAE, the use of GPT-2 generally increases occurrences of positive sentiment for both. Additionally, we conduct human evaluation of AAVE and SAE text generated with GPT-2 to compare contextual rigor and overall quality. △ Less

Submitted 29 October, 2020; v1 submitted 6 October, 2020; originally announced October 2020.

Comments: 7 pages, EMNLP 2020

DRIVE: A Digital Network Oracle for Cooperative Intelligent Transportation Systems

Authors: Ioannis Mavromatis, Robert J. Piechocki, Mahesh Sooriyabandara, Arjun Parekh

Abstract: In a world where Artificial Intelligence revolutionizes inference, prediction and decision-making tasks, Digital Twins emerge as game-changing tools. A case in point is the development and optimization of Cooperative Intelligent Transportation Systems (C-ITSs): a confluence of cyber-physical digital infrastructure and (semi)automated mobility. Herein we introduce Digital Twin for self-dRiving Inte… ▽ More In a world where Artificial Intelligence revolutionizes inference, prediction and decision-making tasks, Digital Twins emerge as game-changing tools. A case in point is the development and optimization of Cooperative Intelligent Transportation Systems (C-ITSs): a confluence of cyber-physical digital infrastructure and (semi)automated mobility. Herein we introduce Digital Twin for self-dRiving Intelligent VEhicles (DRIVE). The developed framework tackles shortcomings of traditional vehicular and network simulators. It provides a flexible, modular, and scalable implementation to ensure large-scale, city-wide experimentation with a moderate computational cost. The defining feature of our Digital Twin is a unique architecture allowing for submission of sequential queries, to which the Digital Twin provides instantaneous responses with the "state of the world", and hence is an Oracle. With such bidirectional interaction with external intelligent agents and realistic mobility traces, DRIVE provides the environment for development, training and optimization of Machine Learning based C-ITS solutions. △ Less

Submitted 7 July, 2020; originally announced July 2020.

Comments: Accepted for publication at IEEE ISCC 2020

Evaluating Transformer-Based Multilingual Text Classification

Authors: Sophie Groenwold, Samhita Honnavalli, Lily Ou, Aesha Parekh, Sharon Levy, Diba Mirza, William Yang Wang

Abstract: As NLP tools become ubiquitous in today's technological landscape, they are increasingly applied to languages with a variety of typological structures. However, NLP research does not focus primarily on typological differences in its analysis of state-of-the-art language models. As a result, NLP tools perform unequally across languages with different syntactic and morphological structures. Through… ▽ More As NLP tools become ubiquitous in today's technological landscape, they are increasingly applied to languages with a variety of typological structures. However, NLP research does not focus primarily on typological differences in its analysis of state-of-the-art language models. As a result, NLP tools perform unequally across languages with different syntactic and morphological structures. Through a detailed discussion of word order typology, morphological typology, and comparative linguistics, we identify which variables most affect language modeling efficacy; in addition, we calculate word order and morphological similarity indices to aid our empirical study. We then use this background to support our analysis of an experiment we conduct using multi-class text classification on eight languages and eight models. △ Less

Submitted 30 April, 2020; v1 submitted 28 April, 2020; originally announced April 2020.

Comments: Total of 15 pages (9 pages for paper, 2 pages for references, 4 pages for appendix). Changed title

A Neural Architecture for Person Ontology population

Authors: Balaji Ganesan, Riddhiman Dasgupta, Akshay Parekh, Hima Patel, Berthold Reinwald

Abstract: A person ontology comprising concepts, attributes and relationships of people has a number of applications in data protection, didentification, population of knowledge graphs for business intelligence and fraud prevention. While artificial neural networks have led to improvements in Entity Recognition, Entity Classification, and Relation Extraction, creating an ontology largely remains a manual pr… ▽ More A person ontology comprising concepts, attributes and relationships of people has a number of applications in data protection, didentification, population of knowledge graphs for business intelligence and fraud prevention. While artificial neural networks have led to improvements in Entity Recognition, Entity Classification, and Relation Extraction, creating an ontology largely remains a manual process, because it requires a fixed set of semantic relations between concepts. In this work, we present a system for automatically populating a person ontology graph from unstructured data using neural models for Entity Classification and Relation Extraction. We introduce a new dataset for these tasks and discuss our results. △ Less

Submitted 22 January, 2020; originally announced January 2020.

Comments: 6 pages, 10 figures. arXiv admin note: substantial text overlap with arXiv:1811.09368

Taxonomical hierarchy of canonicalized relations from multiple Knowledge Bases

Authors: Akshay Parekh, Ashish Anand, Amit Awekar

Abstract: This work addresses two important questions pertinent to Relation Extraction (RE). First, what are all possible relations that could exist between any two given entity types? Second, how do we define an unambiguous taxonomical (is-a) hierarchy among the identified relations? To address the first question, we use three resources Wikipedia Infobox, Wikidata, and DBpedia. This study focuses on relati… ▽ More This work addresses two important questions pertinent to Relation Extraction (RE). First, what are all possible relations that could exist between any two given entity types? Second, how do we define an unambiguous taxonomical (is-a) hierarchy among the identified relations? To address the first question, we use three resources Wikipedia Infobox, Wikidata, and DBpedia. This study focuses on relations between person, organization and location entity types. We exploit Wikidata and DBpedia in a data-driven manner, and Wikipedia Infobox templates manually to generate lists of relations. Further, to address the second question, we canonicalize, filter, and combine the identified relations from the three resources to construct a taxonomical hierarchy. This hierarchy contains 623 canonical relations with highest contribution from Wikipedia Infobox followed by DBpedia and Wikidata. The generated relation list subsumes an average of 85% of relations from RE datasets when entity types are restricted. △ Less

Submitted 12 November, 2019; v1 submitted 13 September, 2019; originally announced September 2019.

Comments: Accepted at CoDS-COMAD 2020

Matching Observations to Distributions: Efficient Estimation via Sparsified Hungarian Algorithm

Authors: Sinho Chewi, Forest Yang, Avishek Ghosh, Abhay Parekh, Kannan Ramchandran

Abstract: Suppose we are given observations, where each observation is drawn independently from one of $k$ known distributions. The goal is to match each observation to the distribution from which it was drawn. We observe that the maximum likelihood estimator (MLE) for this problem can be computed using weighted bipartite matching, even when $n$, the number of observations per distribution, exceeds one. Thi… ▽ More Suppose we are given observations, where each observation is drawn independently from one of $k$ known distributions. The goal is to match each observation to the distribution from which it was drawn. We observe that the maximum likelihood estimator (MLE) for this problem can be computed using weighted bipartite matching, even when $n$, the number of observations per distribution, exceeds one. This is achieved by instantiating $n$ duplicates of each distribution node. However, in the regime where the number of observations per distribution is much larger than the number of distributions, the Hungarian matching algorithm for computing the weighted bipartite matching requires $\mathcal O(n^3)$ time. We introduce a novel randomized matching algorithm that reduces the runtime to $\tilde{\mathcal O}(n^2)$ by sparsifying the original graph, returning the exact MLE with high probability. Next, we give statistical justification for using the MLE by bounding the excess risk of the MLE, where the loss is defined as the negative log-likelihood. We test these bounds for the case of isotropic Gaussians with equal covariances and whose means are separated by a distance $η$, and find (1) that $\gg \log k$ separation suffices to drive the proportion of mismatches of the MLE to 0, and (2) that the expected fraction of mismatched observations goes to zero at rate $\mathcal O({(\log k)}^2/η^2)$. △ Less

Submitted 29 September, 2019; v1 submitted 18 June, 2018; originally announced June 2018.

Comments: 8 pages, 1 figure; to appear in the 57th Annual Allerton Conference on Communication, Control, and Computing

MSC Class: 68W20

Improved Sparse Low-Rank Matrix Estimation

Authors: Ankit Parekh, Ivan W. Selesnick

Abstract: We address the problem of estimating a sparse low-rank matrix from its noisy observation. We propose an objective function consisting of a data-fidelity term and two parameterized non-convex penalty functions. Further, we show how to set the parameters of the non-convex penalty functions, in order to ensure that the objective function is strictly convex. The proposed objective function better esti… ▽ More We address the problem of estimating a sparse low-rank matrix from its noisy observation. We propose an objective function consisting of a data-fidelity term and two parameterized non-convex penalty functions. Further, we show how to set the parameters of the non-convex penalty functions, in order to ensure that the objective function is strictly convex. The proposed objective function better estimates sparse low-rank matrices than a convex method which utilizes the sum of the nuclear norm and the $\ell_1$ norm. We derive an algorithm (as an instance of ADMM) to solve the proposed problem, and guarantee its convergence provided the scalar augmented Lagrangian parameter is set appropriately. We demonstrate the proposed method for denoising an audio signal and an adjacency matrix representing protein interactions in the `Escherichia coli' bacteria. △ Less

Submitted 12 April, 2017; v1 submitted 29 April, 2016; originally announced May 2016.

Comments: 10 pages, 10 figures

Journal ref: Signal Processing, Apr. 2017

Enhanced Low-Rank Matrix Approximation

Authors: Ankit Parekh, Ivan W. Selesnick

Abstract: This letter proposes to estimate low-rank matrices by formulating a convex optimization problem with non-convex regularization. We employ parameterized non-convex penalty functions to estimate the non-zero singular values more accurately than the nuclear norm. A closed-form solution for the global optimum of the proposed objective function (sum of data fidelity and the non-convex regularizer) is a… ▽ More This letter proposes to estimate low-rank matrices by formulating a convex optimization problem with non-convex regularization. We employ parameterized non-convex penalty functions to estimate the non-zero singular values more accurately than the nuclear norm. A closed-form solution for the global optimum of the proposed objective function (sum of data fidelity and the non-convex regularizer) is also derived. The solution reduces to singular value thresholding method as a special case. The proposed method is demonstrated for image denoising. △ Less

Submitted 12 April, 2016; v1 submitted 5 November, 2015; originally announced November 2015.

Comments: 5 pages, 2 figures. MATLAB code available at https://goo.gl/xAi85N

Journal ref: IEEE Signal Processing Letters, vol. 23, no. 4, pp.493-497, Apr. 2016

The Square Root Agreement Rule for Incentivizing Truthful Feedback on Online Platforms

Authors: Vijay Kamble, Nihar Shah, David Marn, Abhay Parekh, Kannan Ramachandran

Abstract: A major challenge in obtaining evaluations of products or services on e-commerce platforms is eliciting informative responses in the absence of verifiability. This paper proposes the Square Root Agreement Rule (SRA): a simple reward mechanism that incentivizes truthful responses to objective evaluations on such platforms. In this mechanism, an agent gets a reward for an evaluation only if her answ… ▽ More A major challenge in obtaining evaluations of products or services on e-commerce platforms is eliciting informative responses in the absence of verifiability. This paper proposes the Square Root Agreement Rule (SRA): a simple reward mechanism that incentivizes truthful responses to objective evaluations on such platforms. In this mechanism, an agent gets a reward for an evaluation only if her answer matches that of her peer, where this reward is inversely proportional to a popularity index of the answer. This index is defined to be the square root of the empirical frequency at which any two agents performing the same evaluation agree on the particular answer across evaluations of similar entities operating on the platform. Rarely agreed-upon answers thus earn a higher reward than answers for which agreements are relatively more common. We show that in the many tasks regime, the truthful equilibrium under SRA is strictly payoff-dominant across large classes of natural equilibria that could arise in these settings, thus increasing the likelihood of its adoption. While there exist other mechanisms achieving such guarantees, they either impose additional assumptions on the response distribution that are not generally satisfied for objective evaluations or they incentivize truthful behavior only if each agent performs a prohibitively large number of evaluations and commits to using the same strategy for each evaluation. SRA is the first known incentive mechanism satisfying such guarantees without imposing any such requirements. Moreover, our empirical findings demonstrate the robustness of the incentive properties of SRA in the presence of mild subjectivity or observational biases in the responses. These properties make SRA uniquely attractive for administering reward-based incentive schemes (e.g., rebates, discounts, reputation scores, etc.) on online platforms. △ Less

Submitted 24 February, 2022; v1 submitted 24 July, 2015; originally announced July 2015.

Comments: To appear in Management Science (2022)

Estimation from Pairwise Comparisons: Sharp Minimax Bounds with Topology Dependence

Authors: Nihar B. Shah, Sivaraman Balakrishnan, Joseph Bradley, Abhay Parekh, Kannan Ramchandran, Martin J. Wainwright

Abstract: Data in the form of pairwise comparisons arises in many domains, including preference elicitation, sporting competitions, and peer grading among others. We consider parametric ordinal models for such pairwise comparison data involving a latent vector $w^* \in \mathbb{R}^d$ that represents the "qualities" of the $d$ items being compared; this class of models includes the two most widely used parame… ▽ More Data in the form of pairwise comparisons arises in many domains, including preference elicitation, sporting competitions, and peer grading among others. We consider parametric ordinal models for such pairwise comparison data involving a latent vector $w^* \in \mathbb{R}^d$ that represents the "qualities" of the $d$ items being compared; this class of models includes the two most widely used parametric models--the Bradley-Terry-Luce (BTL) and the Thurstone models. Working within a standard minimax framework, we provide tight upper and lower bounds on the optimal error in estimating the quality score vector $w^*$ under this class of models. The bounds depend on the topology of the comparison graph induced by the subset of pairs being compared via its Laplacian spectrum. Thus, in settings where the subset of pairs may be chosen, our results provide principled guidelines for making this choice. Finally, we compare these error rates to those under cardinal measurement models and show that the error rates in the ordinal and cardinal settings have identical scalings apart from constant pre-factors. △ Less

Submitted 6 May, 2015; originally announced May 2015.

Comments: 39 pages, 5 figures. Significant extension of arXiv:1406.6618

Convex Denoising using Non-Convex Tight Frame Regularization

Authors: Ankit Parekh, Ivan W. Selesnick

Abstract: This paper considers the problem of signal denoising using a sparse tight-frame analysis prior. The L1 norm has been extensively used as a regularizer to promote sparsity; however, it tends to under-estimate non-zero values of the underlying signal. To more accurately estimate non-zero values, we propose the use of a non-convex regularizer, chosen so as to ensure convexity of the objective functio… ▽ More This paper considers the problem of signal denoising using a sparse tight-frame analysis prior. The L1 norm has been extensively used as a regularizer to promote sparsity; however, it tends to under-estimate non-zero values of the underlying signal. To more accurately estimate non-zero values, we propose the use of a non-convex regularizer, chosen so as to ensure convexity of the objective function. The convexity of the objective function is ensured by constraining the parameter of the non-convex penalty. We use ADMM to obtain a solution and show how to guarantee that ADMM converges to the global optimum of the objective function. We illustrate the proposed method for 1D and 2D signal denoising. △ Less

Submitted 3 June, 2015; v1 submitted 3 April, 2015; originally announced April 2015.

Comments: 5 pages, 6 figures

Journal ref: IEEE Signal Processing Letters, 22(10):1786-1790, Oct. 2015

When is it Better to Compare than to Score?

Authors: Nihar B. Shah, Sivaraman Balakrishnan, Joseph Bradley, Abhay Parekh, Kannan Ramchandran, Martin Wainwright

Abstract: When eliciting judgements from humans for an unknown quantity, one often has the choice of making direct-scoring (cardinal) or comparative (ordinal) measurements. In this paper we study the relative merits of either choice, providing empirical and theoretical guidelines for the selection of a measurement scheme. We provide empirical evidence based on experiments on Amazon Mechanical Turk that in a… ▽ More When eliciting judgements from humans for an unknown quantity, one often has the choice of making direct-scoring (cardinal) or comparative (ordinal) measurements. In this paper we study the relative merits of either choice, providing empirical and theoretical guidelines for the selection of a measurement scheme. We provide empirical evidence based on experiments on Amazon Mechanical Turk that in a variety of tasks, (pairwise-comparative) ordinal measurements have lower per sample noise and are typically faster to elicit than cardinal ones. Ordinal measurements however typically provide less information. We then consider the popular Thurstone and Bradley-Terry-Luce (BTL) models for ordinal measurements and characterize the minimax error rates for estimating the unknown quantity. We compare these minimax error rates to those under cardinal measurement models and quantify for what noise levels ordinal measurements are better. Finally, we revisit the data collected from our experiments and show that fitting these models confirms this prediction: for tasks where the noise in ordinal measurements is sufficiently low, the ordinal approach results in smaller errors in the estimation. △ Less

Submitted 25 June, 2014; originally announced June 2014.

An Adaptive Multi-channel P2P Video-on-Demand System using Plug-and-Play Helpers

Authors: Hao Zhang, Minghua Chen, Abhay Parekh, Kannan Ramchandran

Abstract: We present a multi-channel P2P Video-on-Demand (VoD) system using "plug-and-play" helpers. Helpers are heterogenous "micro-servers" with limited storage, bandwidth and number of users they can serve simultaneously. Our proposed system has the following salient features: (1) it minimizes the server load; (2) it is distributed, and requires little or no maintenance overhead and which can easily adap… ▽ More We present a multi-channel P2P Video-on-Demand (VoD) system using "plug-and-play" helpers. Helpers are heterogenous "micro-servers" with limited storage, bandwidth and number of users they can serve simultaneously. Our proposed system has the following salient features: (1) it minimizes the server load; (2) it is distributed, and requires little or no maintenance overhead and which can easily adapt to system dynamics; and (3) it is adaptable to varying supply and demand patterns across multiple video channels irrespective of video popularity. Our proposed solution jointly optimizes over helper-user topology, video storage allocation and bandwidth allocation. The combinatorial nature of the problem and the system demand for distributed algorithms makes the problem uniquely challenging. By utilizing Lagrangian decomposition and Markov chain approximation based arguments, we address this challenge by designing two distributed algorithms running in tandem: a primal-dual storage and bandwidth allocation algorithm and a "soft-worst-neighbor-choking" topology-building algorithm. Our scheme provably converges to a near-optimal solution, and is easy to implement in practice. Simulation results validate that the proposed scheme achieves minimum sever load under highly heterogeneous combinations of supply and demand patterns, and is robust to system dynamics of user/helper churn, user/helper asynchrony, and random delays in the network. △ Less

Submitted 24 November, 2010; originally announced November 2010.

The Approximate Capacity of the Many-to-One and One-to-Many Gaussian Interference Channels

Authors: Guy Bresler, Abhay Parekh, David Tse

Abstract: Recently, Etkin, Tse, and Wang found the capacity region of the two-user Gaussian interference channel to within one bit/s/Hz. A natural goal is to apply this approach to the Gaussian interference channel with an arbitrary number of users. We make progress towards this goal by finding the capacity region of the many-to-one and one-to-many Gaussian interference channels to within a constant numbe… ▽ More Recently, Etkin, Tse, and Wang found the capacity region of the two-user Gaussian interference channel to within one bit/s/Hz. A natural goal is to apply this approach to the Gaussian interference channel with an arbitrary number of users. We make progress towards this goal by finding the capacity region of the many-to-one and one-to-many Gaussian interference channels to within a constant number of bits. The result makes use of a deterministic model to provide insight into the Gaussian channel. The deterministic model makes explicit the dimension of signal scale. A central theme emerges: the use of lattice codes for alignment of interfering signals on the signal scale. △ Less

Submitted 21 September, 2008; originally announced September 2008.

Comments: 45 pages, 16 figures. Submitted to IEEE Transactions on Information Theory