-
Llamas Know What GPTs Don't Show: Surrogate Models for Confidence Estimation
Authors:
Vaishnavi Shrivastava,
Percy Liang,
Ananya Kumar
Abstract:
To maintain user trust, large language models (LLMs) should signal low confidence on examples where they are incorrect, instead of misleading the user. The standard approach of estimating confidence is to use the softmax probabilities of these models, but as of November 2023, state-of-the-art LLMs such as GPT-4 and Claude-v1.3 do not provide access to these probabilities. We first study eliciting…
▽ More
To maintain user trust, large language models (LLMs) should signal low confidence on examples where they are incorrect, instead of misleading the user. The standard approach of estimating confidence is to use the softmax probabilities of these models, but as of November 2023, state-of-the-art LLMs such as GPT-4 and Claude-v1.3 do not provide access to these probabilities. We first study eliciting confidence linguistically -- asking an LLM for its confidence in its answer -- which performs reasonably (80.5% AUC on GPT-4 averaged across 12 question-answering datasets -- 7% above a random baseline) but leaves room for improvement. We then explore using a surrogate confidence model -- using a model where we do have probabilities to evaluate the original model's confidence in a given question. Surprisingly, even though these probabilities come from a different and often weaker model, this method leads to higher AUC than linguistic confidences on 9 out of 12 datasets. Our best method composing linguistic confidences and surrogate model probabilities gives state-of-the-art confidence estimates on all 12 datasets (84.6% average AUC on GPT-4).
△ Less
Submitted 15 November, 2023;
originally announced November 2023.
-
Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs
Authors:
Shashank Gupta,
Vaishnavi Shrivastava,
Ameet Deshpande,
Ashwin Kalyan,
Peter Clark,
Ashish Sabharwal,
Tushar Khot
Abstract:
Recent works have showcased the ability of LLMs to embody diverse personas in their responses, exemplified by prompts like 'You are Yoda. Explain the Theory of Relativity.' While this ability allows personalization of LLMs and enables human behavior simulation, its effect on LLMs' capabilities remains unclear. To fill this gap, we present the first extensive study of the unintended side-effects of…
▽ More
Recent works have showcased the ability of LLMs to embody diverse personas in their responses, exemplified by prompts like 'You are Yoda. Explain the Theory of Relativity.' While this ability allows personalization of LLMs and enables human behavior simulation, its effect on LLMs' capabilities remains unclear. To fill this gap, we present the first extensive study of the unintended side-effects of persona assignment on the ability of LLMs to perform basic reasoning tasks. Our study covers 24 reasoning datasets, 4 LLMs, and 19 diverse personas (e.g. an Asian person) spanning 5 socio-demographic groups. Our experiments unveil that LLMs harbor deep rooted bias against various socio-demographics underneath a veneer of fairness. While they overtly reject stereotypes when explicitly asked ('Are Black people less skilled at mathematics?'), they manifest stereotypical and erroneous presumptions when asked to answer questions while adopting a persona. These can be observed as abstentions in responses, e.g., 'As a Black person, I can't answer this question as it requires math knowledge', and generally result in a substantial performance drop. Our experiments with ChatGPT-3.5 show that this bias is ubiquitous - 80% of our personas demonstrate bias; it is significant - some datasets show performance drops of 70%+; and can be especially harmful for certain groups - some personas suffer statistically significant drops on 80%+ of the datasets. Overall, all 4 LLMs exhibit this bias to varying extents, with GPT-4-Turbo showing the least but still a problematic amount of bias (evident in 42% of the personas). Further analysis shows that these persona-induced errors can be hard-to-discern and hard-to-avoid. Our findings serve as a cautionary tale that the practice of assigning personas to LLMs - a trend on the rise - can surface their deep-rooted biases and have unforeseeable and detrimental side-effects.
△ Less
Submitted 27 January, 2024; v1 submitted 8 November, 2023;
originally announced November 2023.
-
Benchmarking and Improving Generator-Validator Consistency of Language Models
Authors:
Xiang Lisa Li,
Vaishnavi Shrivastava,
Siyan Li,
Tatsunori Hashimoto,
Percy Liang
Abstract:
As of September 2023, ChatGPT correctly answers "what is 7+8" with 15, but when asked "7+8=15, True or False" it responds with "False". This inconsistency between generating and validating an answer is prevalent in language models (LMs) and erodes trust. In this paper, we propose a framework for measuring the consistency between generation and validation (which we call generator-validator consiste…
▽ More
As of September 2023, ChatGPT correctly answers "what is 7+8" with 15, but when asked "7+8=15, True or False" it responds with "False". This inconsistency between generating and validating an answer is prevalent in language models (LMs) and erodes trust. In this paper, we propose a framework for measuring the consistency between generation and validation (which we call generator-validator consistency, or GV-consistency), finding that even GPT-4, a state-of-the-art LM, is GV-consistent only 76% of the time. To improve the consistency of LMs, we propose to finetune on the filtered generator and validator responses that are GV-consistent, and call this approach consistency fine-tuning. We find that this approach improves GV-consistency of Alpaca-30B from 60% to 93%, and the improvement extrapolates to unseen tasks and domains (e.g., GV-consistency for positive style transfers extrapolates to unseen styles like humor). In addition to improving consistency, consistency fine-tuning improves both generator quality and validator accuracy without using any labeled data. Evaluated across 6 tasks, including math questions, knowledge-intensive QA, and instruction following, our method improves the generator quality by 16% and the validator accuracy by 6.3% across all tasks.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
Exploring Low-Cost Transformer Model Compression for Large-Scale Commercial Reply Suggestions
Authors:
Vaishnavi Shrivastava,
Radhika Gaonkar,
Shashank Gupta,
Abhishek Jha
Abstract:
Fine-tuning pre-trained language models improves the quality of commercial reply suggestion systems, but at the cost of unsustainable training times. Popular training time reduction approaches are resource intensive, thus we explore low-cost model compression techniques like Layer Drop** and Layer Freezing. We demonstrate the efficacy of these techniques in large-data scenarios, enabling the tra…
▽ More
Fine-tuning pre-trained language models improves the quality of commercial reply suggestion systems, but at the cost of unsustainable training times. Popular training time reduction approaches are resource intensive, thus we explore low-cost model compression techniques like Layer Drop** and Layer Freezing. We demonstrate the efficacy of these techniques in large-data scenarios, enabling the training time reduction for a commercial email reply suggestion system by 42%, without affecting the model relevance or user engagement. We further study the robustness of these techniques to pre-trained model and dataset size ablation, and share several insights and recommendations for commercial applications.
△ Less
Submitted 27 November, 2021;
originally announced November 2021.
-
UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis
Authors:
Fatemehsadat Mireshghallah,
Vaishnavi Shrivastava,
Milad Shokouhi,
Taylor Berg-Kirkpatrick,
Robert Sim,
Dimitrios Dimitriadis
Abstract:
Global models are trained to be as generalizable as possible, with user invariance considered desirable since the models are shared across multitudes of users. As such, these models are often unable to produce personalized responses for individual users, based on their data. Contrary to widely-used personalization techniques based on few-shot learning, we propose UserIdentifier, a novel scheme for…
▽ More
Global models are trained to be as generalizable as possible, with user invariance considered desirable since the models are shared across multitudes of users. As such, these models are often unable to produce personalized responses for individual users, based on their data. Contrary to widely-used personalization techniques based on few-shot learning, we propose UserIdentifier, a novel scheme for training a single shared model for all users. Our approach produces personalized responses by adding fixed, non-trainable user identifiers to the input data. We empirically demonstrate that this proposed method outperforms the prefix-tuning based state-of-the-art approach by up to 13%, on a suite of sentiment analysis datasets. We also show that, unlike prior work, this method needs neither any additional model parameters nor any extra rounds of few-shot fine-tuning.
△ Less
Submitted 3 May, 2022; v1 submitted 30 September, 2021;
originally announced October 2021.
-
Grou** Search Results with Product Graphs in E-commerce Platforms
Authors:
Suhas Ranganath,
Shibsankar Das,
Sanjay Thilaivasan,
Shipra Agarwal,
Varun Shrivastava
Abstract:
Showing relevant search results to the user is the primary challenge for any search system. Walmart e-commerce provides an omnichannel search platform to its customers to search from millions of products. This search platform takes a textual query as input and shows relevant items from the catalog. One of the primary challenges is that this queries are complex to understand as it contains multiple…
▽ More
Showing relevant search results to the user is the primary challenge for any search system. Walmart e-commerce provides an omnichannel search platform to its customers to search from millions of products. This search platform takes a textual query as input and shows relevant items from the catalog. One of the primary challenges is that this queries are complex to understand as it contains multiple intent in many cases. This paper proposes a framework to group search results into multiple ranked lists intending to provide better user intent. The framework is to create a product graph having relations between product entities and utilize it to group search results into a series of stacks where each stack provides a group of items based on a precise intent. As an example, for a query "milk," the results can be grouped into multiple stacks of "white milk", "low-fat milk", "almond milk", "flavored milk". We measure the impact of our algorithm by evaluating how it improves the user experience both in terms of search quality relevance and user behavioral signals like Add-To-Cart.
△ Less
Submitted 20 September, 2021;
originally announced September 2021.
-
Identifying Linked Fraudulent Activities Using GraphConvolution Network
Authors:
Sharmin Pathan,
Vyom Shrivastava
Abstract:
In this paper, we present a novel approach to identify linked fraudulent activities or actors sharing similar attributes, using Graph Convolution Network (GCN). These linked fraudulent activities can be visualized as graphs with abstract concepts like relationships and interactions, which makes GCNs an ideal solution to identify the graph edges which serve as links between fraudulent nodes. Tradit…
▽ More
In this paper, we present a novel approach to identify linked fraudulent activities or actors sharing similar attributes, using Graph Convolution Network (GCN). These linked fraudulent activities can be visualized as graphs with abstract concepts like relationships and interactions, which makes GCNs an ideal solution to identify the graph edges which serve as links between fraudulent nodes. Traditional approaches like community detection require strong links between fraudulent attempts like shared attributes to find communities and the supervised solutions require large amount of training data which may not be available in fraud scenarios and work best to provide binary separation between fraudulent and non fraudulent activities. Our approach overcomes the drawbacks of traditional methods as GCNs simply learn similarities between fraudulent nodes to identify clusters of similar attempts and require much smaller dataset to learn. We demonstrate our results on linked accounts with both strong and weak links to identify fraud rings with high confidence. Our results outperform label propagation community detection and supervised GBTs algorithms in terms of solution quality and computation time.
△ Less
Submitted 5 June, 2021;
originally announced June 2021.
-
Reinforcement Learning for Assignment Problem with Time Constraints
Authors:
Sharmin Pathan,
Vyom Shrivastava
Abstract:
We present an end-to-end framework for the Assignment Problem with multiple tasks mapped to a group of workers, using reinforcement learning while preserving many constraints. Tasks and workers have time constraints and there is a cost associated with assigning a worker to a task. Each worker can perform multiple tasks until it exhausts its allowed time units (capacity). We train a reinforcement l…
▽ More
We present an end-to-end framework for the Assignment Problem with multiple tasks mapped to a group of workers, using reinforcement learning while preserving many constraints. Tasks and workers have time constraints and there is a cost associated with assigning a worker to a task. Each worker can perform multiple tasks until it exhausts its allowed time units (capacity). We train a reinforcement learning agent to find near optimal solutions to the problem by minimizing total cost associated with the assignments while maintaining hard constraints. We use proximal policy optimization to optimize model parameters. The model generates a sequence of actions in real-time which correspond to task assignment to workers, without having to retrain for changes in the dynamic state of the environment. In our problem setting reward is computed as negative of the assignment cost. We also demonstrate our results on bin packing and capacitated vehicle routing problem, using the same framework. Our results outperform Google OR-Tools using MIP and CP-SAT solvers with large problem instances, in terms of solution quality and computation time.
△ Less
Submitted 5 June, 2021;
originally announced June 2021.
-
Analysis of Attacks on Hybrid DWT-DCT Algorithm for Digital Image Watermarking With MATLAB
Authors:
Lalit Kumar Saini,
Vishal Shrivastava
Abstract:
Watermarking algorithms needs properties of robustness and perceptibility. But these properties are affected by different -2 types of attacks performed on watermarked images. The goal of performing attacks is destroy the information of watermark hidden in the watermarked image. So every Algorithms should be previously tested by developers so that it would not affected by attacks.
Watermarking algorithms needs properties of robustness and perceptibility. But these properties are affected by different -2 types of attacks performed on watermarked images. The goal of performing attacks is destroy the information of watermark hidden in the watermarked image. So every Algorithms should be previously tested by developers so that it would not affected by attacks.
△ Less
Submitted 17 July, 2014;
originally announced July 2014.
-
A Survey of Digital Watermarking Techniques and its Applications
Authors:
Lalit Kumar Saini,
Vishal Shrivastava
Abstract:
Digital media is the need of a people now a day as the alternate of paper media.As the technology grown up digital media required protection while transferring through internet or others mediums.Watermarking techniques have been developed to fulfill this requirement.This paper aims to provide a detailed survey of all watermarking techniques specially focuses on image watermarking types and its app…
▽ More
Digital media is the need of a people now a day as the alternate of paper media.As the technology grown up digital media required protection while transferring through internet or others mediums.Watermarking techniques have been developed to fulfill this requirement.This paper aims to provide a detailed survey of all watermarking techniques specially focuses on image watermarking types and its applications in today world.
△ Less
Submitted 17 July, 2014;
originally announced July 2014.
-
Artificial Neural Network Based Optical Character Recognition
Authors:
Vivek Shrivastava,
Navdeep Sharma
Abstract:
Optical Character Recognition deals in recognition and classification of characters from an image. For the recognition to be accurate, certain topological and geometrical properties are calculated, based on which a character is classified and recognized. Also, the Human psychology perceives characters by its overall shape and features such as strokes, curves, protrusions, enclosures etc. These pro…
▽ More
Optical Character Recognition deals in recognition and classification of characters from an image. For the recognition to be accurate, certain topological and geometrical properties are calculated, based on which a character is classified and recognized. Also, the Human psychology perceives characters by its overall shape and features such as strokes, curves, protrusions, enclosures etc. These properties, also called Features are extracted from the image by means of spatial pixel-based calculation. A collection of such features, called Vectors, help in defining a character uniquely, by means of an Artificial Neural Network that uses these Feature Vectors.
△ Less
Submitted 19 November, 2012;
originally announced November 2012.
-
Distributed Agile Software Development: A Review
Authors:
Suprika Vasudeva Shrivastava,
Hema Date
Abstract:
Distribution of software development is becoming more and more common in order to save the production cost and reduce the time to market. Large geographical distance, different time zones and cultural differences in distributed software development (DSD) leads to weak communication which adversely affects the project. Using agile practices for distributed development is also gaining momentum in va…
▽ More
Distribution of software development is becoming more and more common in order to save the production cost and reduce the time to market. Large geographical distance, different time zones and cultural differences in distributed software development (DSD) leads to weak communication which adversely affects the project. Using agile practices for distributed development is also gaining momentum in various organizations to increase the quality and performance of the project. This paper explores the intersection of these two significant trends for software development i.e. DSD and agile. We discuss the challenges faced by geographically distributed agile teams and proven practices to address these issues, which will help in building a successful distributed team.
△ Less
Submitted 10 June, 2010;
originally announced June 2010.
-
FP-tree and COFI Based Approach for Mining of Multiple Level Association Rules in Large Databases
Authors:
Virendra Kumar Shrivastava,
Parveen Kumar,
K. R. Pardasani
Abstract:
In recent years, discovery of association rules among itemsets in a large database has been described as an important database-mining problem. The problem of discovering association rules has received considerable research attention and several algorithms for mining frequent itemsets have been developed. Many algorithms have been proposed to discover rules at single concept level. However, mining…
▽ More
In recent years, discovery of association rules among itemsets in a large database has been described as an important database-mining problem. The problem of discovering association rules has received considerable research attention and several algorithms for mining frequent itemsets have been developed. Many algorithms have been proposed to discover rules at single concept level. However, mining association rules at multiple concept levels may lead to the discovery of more specific and concrete knowledge from data. The discovery of multiple level association rules is very much useful in many applications. In most of the studies for multiple level association rule mining, the database is scanned repeatedly which affects the efficiency of mining process. In this research paper, a new method for discovering multilevel association rules is proposed. It is based on FP-tree structure and uses cooccurrence frequent item tree to find frequent items in multilevel concept hierarchy.
△ Less
Submitted 9 March, 2010;
originally announced March 2010.
