-
RE-GAINS & EnChAnT: Intelligent Tool Manipulation Systems For Enhanced Query Responses
Authors:
Sahil Girhepuje,
Siva Sankar Sajeev,
Purvam Jain,
Arya Sikder,
Adithya Rama Varma,
Ryan George,
Akshay Govind Srinivasan,
Mahendra Kurup,
Ashmit Sinha,
Sudip Mondal
Abstract:
Large Language Models (LLMs) currently struggle with tool invocation and chaining, as they often hallucinate or miss essential steps in a sequence. We propose RE-GAINS and EnChAnT, two novel frameworks that empower LLMs to tackle complex user queries by making API calls to external tools based on tool descriptions and argument lists. Tools are chained based on the expected output, without receivin…
▽ More
Large Language Models (LLMs) currently struggle with tool invocation and chaining, as they often hallucinate or miss essential steps in a sequence. We propose RE-GAINS and EnChAnT, two novel frameworks that empower LLMs to tackle complex user queries by making API calls to external tools based on tool descriptions and argument lists. Tools are chained based on the expected output, without receiving the actual results from each individual call. EnChAnT, an open-source solution, leverages an LLM format enforcer, OpenChat 3.5 (an LLM), and ToolBench's API Retriever. RE-GAINS utilizes OpenAI models and embeddings with a specialized prompt based on the $\underline{R}$easoning vi$\underline{a}$ $\underline{P}$lanning $(RAP)$ framework. Both frameworks are low cost (0.01\$ per query). Our key contribution is enabling LLMs for tool invocation and chaining using modifiable, externally described tools.
△ Less
Submitted 20 June, 2024; v1 submitted 28 January, 2024;
originally announced January 2024.
-
Magnetotransport Properties of Epitaxial Films and Hall Bar Devices of the Correlated Layered Ruthenate Sr$_3$Ru$_2$O$_7$
Authors:
Prosper Ngabonziza,
Anand Sharma,
Anna Scheid,
Sethulakshmi Sajeev,
Peter A. van Aken,
Jochen Mannhart
Abstract:
For epitaxial Sr$_3$Ru$_2$O$_7$ films grown by pulsed laser deposition, we report a combined structural and magnetotransport study of thin films and Hall bar devices patterned side-by-side on the same film. Structural properties of these films are investigated using X-ray diffraction and high-resolution transmission electron microscopy, and confirm that these films are epitaxially oriented and nea…
▽ More
For epitaxial Sr$_3$Ru$_2$O$_7$ films grown by pulsed laser deposition, we report a combined structural and magnetotransport study of thin films and Hall bar devices patterned side-by-side on the same film. Structural properties of these films are investigated using X-ray diffraction and high-resolution transmission electron microscopy, and confirm that these films are epitaxially oriented and nearly phase pure. For magnetic fields applied along the $c-$axis, a positive magnetoresistance of 10\% is measured for unpatterned Sr$_3$Ru$_2$O$_7$ films, whereas for patterned Hall bar devices of channel widths of $10$ and $5\, μ$m, magnetoresistance values of 40\% and 140\% are found, respectively. These films show switching behaviors from positive to negative magnetoresistance that are controlled by the direction of the applied magnetic field. The present results provide a promising route for achieving stable epitaxial synthesis of intermediate members of correlated layered strontium ruthenates, and for the exploration of device physics in thin films of these compounds.
△ Less
Submitted 18 March, 2024; v1 submitted 13 October, 2023;
originally announced October 2023.
-
Rule By Example: Harnessing Logical Rules for Explainable Hate Speech Detection
Authors:
Christopher Clarke,
Matthew Hall,
Gaurav Mittal,
Ye Yu,
Sandra Sajeev,
Jason Mars,
Mei Chen
Abstract:
Classic approaches to content moderation typically apply a rule-based heuristic approach to flag content. While rules are easily customizable and intuitive for humans to interpret, they are inherently fragile and lack the flexibility or robustness needed to moderate the vast amount of undesirable content found online today. Recent advances in deep learning have demonstrated the promise of using hi…
▽ More
Classic approaches to content moderation typically apply a rule-based heuristic approach to flag content. While rules are easily customizable and intuitive for humans to interpret, they are inherently fragile and lack the flexibility or robustness needed to moderate the vast amount of undesirable content found online today. Recent advances in deep learning have demonstrated the promise of using highly effective deep neural models to overcome these challenges. However, despite the improved performance, these data-driven models lack transparency and explainability, often leading to mistrust from everyday users and a lack of adoption by many platforms. In this paper, we present Rule By Example (RBE): a novel exemplar-based contrastive learning approach for learning from logical rules for the task of textual content moderation. RBE is capable of providing rule-grounded predictions, allowing for more explainable and customizable predictions compared to typical deep learning-based approaches. We demonstrate that our approach is capable of learning rich rule embedding representations using only a few data examples. Experimental results on 3 popular hate speech classification datasets show that RBE is able to outperform state-of-the-art deep learning classifiers as well as the use of rules in both supervised and unsupervised settings while providing explainable model predictions via rule-grounding.
△ Less
Submitted 24 July, 2023;
originally announced July 2023.
-
Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modality
Authors:
Jialin Yuan,
Ye Yu,
Gaurav Mittal,
Matthew Hall,
Sandra Sajeev,
Mei Chen
Abstract:
There is a rapidly growing need for multimodal content moderation (CM) as more and more content on social media is multimodal in nature. Existing unimodal CM systems may fail to catch harmful content that crosses modalities (e.g., memes or videos), which may lead to severe consequences. In this paper, we present a novel CM model, Asymmetric Mixed-Modal Moderation (AM3), to target multimodal and un…
▽ More
There is a rapidly growing need for multimodal content moderation (CM) as more and more content on social media is multimodal in nature. Existing unimodal CM systems may fail to catch harmful content that crosses modalities (e.g., memes or videos), which may lead to severe consequences. In this paper, we present a novel CM model, Asymmetric Mixed-Modal Moderation (AM3), to target multimodal and unimodal CM tasks. Specifically, to address the asymmetry in semantics between vision and language, AM3 has a novel asymmetric fusion architecture that is designed to not only fuse the common knowledge in both modalities but also to exploit the unique information in each modality. Unlike previous works that focus on representing the two modalities into a similar feature space while overlooking the intrinsic difference between the information conveyed in multimodality and in unimodality (asymmetry in modalities), we propose a novel cross-modality contrastive loss to learn the unique knowledge that only appears in multimodality. This is critical as some harmful intent may only be conveyed through the intersection of both modalities. With extensive experiments, we show that AM3 outperforms all existing state-of-the-art methods on both multimodal and unimodal CM benchmarks.
△ Less
Submitted 13 December, 2023; v1 submitted 17 May, 2023;
originally announced May 2023.
-
Secure and Ultra-Reliable Provenance Recovery in Sparse Networks: Strategies and Performance Bounds
Authors:
Suraj Sajeev,
Manish Bansal,
Sriraam S V,
J. Harshan,
Huzur Saran,
Yih-Chun Hu
Abstract:
Provenance embedding algorithms are well known for tracking the footprints of information flow in wireless networks. Recently, low-latency provenance embedding algorithms have received traction in vehicular networks owing to strict deadlines on the delivery of packets. While existing low-latency provenance embedding methods focus on reducing the packet delay, they assume a complete graph on the un…
▽ More
Provenance embedding algorithms are well known for tracking the footprints of information flow in wireless networks. Recently, low-latency provenance embedding algorithms have received traction in vehicular networks owing to strict deadlines on the delivery of packets. While existing low-latency provenance embedding methods focus on reducing the packet delay, they assume a complete graph on the underlying topology due to the mobility of the participating nodes. We identify that the complete graph assumption leads to sub-optimal performance in provenance recovery, especially when the vehicular network is sparse, which is usually observed outside peak-hour traffic conditions. As a result, we propose a two-part approach to design provenance embedding algorithms for sparse vehicular networks. In the first part, we propose secure and practical topology-learning strategies, whereas in the second part, we design provenance embedding algorithms that guarantee ultra-reliability by incorporating the topology knowledge at the destination during the provenance recovery process. Besides the novel idea of using topology knowledge for provenance recovery, a distinguishing feature for achieving ultra-reliability is the use of hash-chains in the packet, which trade communication-overhead of the packet with the complexity-overhead at the destination. We derive tight upper bounds on the performance of our strategies, and show that the derived bounds, when optimized with appropriate constraints, deliver design parameters that outperform existing methods. Finally, we also implement our ideas on OMNeT++ based simulation environment to show that their latency benefits indeed make them suitable for vehicular network applications.
△ Less
Submitted 31 March, 2022;
originally announced April 2022.
-
Contextual Bandit Applications in Customer Support Bot
Authors:
Sandra Sajeev,
Jade Huang,
Nikos Karampatziakis,
Matthew Hall,
Sebastian Kochman,
Weizhu Chen
Abstract:
Virtual support agents have grown in popularity as a way for businesses to provide better and more accessible customer service. Some challenges in this domain include ambiguous user queries as well as changing support topics and user behavior (non-stationarity). We do, however, have access to partial feedback provided by the user (clicks, surveys, and other events) which can be leveraged to improv…
▽ More
Virtual support agents have grown in popularity as a way for businesses to provide better and more accessible customer service. Some challenges in this domain include ambiguous user queries as well as changing support topics and user behavior (non-stationarity). We do, however, have access to partial feedback provided by the user (clicks, surveys, and other events) which can be leveraged to improve the user experience. Adaptable learning techniques, like contextual bandits, are a natural fit for this problem setting. In this paper, we discuss real-world implementations of contextual bandits (CB) for the Microsoft virtual agent. It includes intent disambiguation based on neural-linear bandits (NLB) and contextual recommendations based on a collection of multi-armed bandits (MAB). Our solutions have been deployed to production and have improved key business metrics of the Microsoft virtual agent, as confirmed by A/B experiments. Results include a relative increase of over 12% in problem resolution rate and relative decrease of over 4% in escalations to a human operator. While our current use cases focus on intent disambiguation and contextual recommendation for support bots, we believe our methods can be extended to other domains.
△ Less
Submitted 6 December, 2021;
originally announced December 2021.
-
CoDA: Contrast-enhanced and Diversity-promoting Data Augmentation for Natural Language Understanding
Authors:
Yanru Qu,
Dinghan Shen,
Yelong Shen,
Sandra Sajeev,
Jiawei Han,
Weizhu Chen
Abstract:
Data augmentation has been demonstrated as an effective strategy for improving model generalization and data efficiency. However, due to the discrete nature of natural language, designing label-preserving transformations for text data tends to be more challenging. In this paper, we propose a novel data augmentation framework dubbed CoDA, which synthesizes diverse and informative augmented examples…
▽ More
Data augmentation has been demonstrated as an effective strategy for improving model generalization and data efficiency. However, due to the discrete nature of natural language, designing label-preserving transformations for text data tends to be more challenging. In this paper, we propose a novel data augmentation framework dubbed CoDA, which synthesizes diverse and informative augmented examples by integrating multiple transformations organically. Moreover, a contrastive regularization objective is introduced to capture the global relationship among all the data samples. A momentum encoder along with a memory bank is further leveraged to better estimate the contrastive loss. To verify the effectiveness of the proposed framework, we apply CoDA to Transformer-based models on a wide range of natural language understanding tasks. On the GLUE benchmark, CoDA gives rise to an average improvement of 2.2% while applied to the RoBERTa-large model. More importantly, it consistently exhibits stronger results relative to several competitive data augmentation and adversarial training base-lines (including the low-resource settings). Extensive experiments show that the proposed contrastive objective can be flexibly combined with various data augmentation approaches to further boost their performance, highlighting the wide applicability of the CoDA framework.
△ Less
Submitted 16 October, 2020;
originally announced October 2020.
-
xBD: A Dataset for Assessing Building Damage from Satellite Imagery
Authors:
Ritwik Gupta,
Richard Hosfelt,
Sandra Sajeev,
Nirav Patel,
Bryce Goodman,
Jigar Doshi,
Eric Heim,
Howie Choset,
Matthew Gaston
Abstract:
We present xBD, a new, large-scale dataset for the advancement of change detection and building damage assessment for humanitarian assistance and disaster recovery research. Natural disaster response requires an accurate understanding of damaged buildings in an affected region. Current response strategies require in-person damage assessments within 24-48 hours of a disaster. Massive potential exis…
▽ More
We present xBD, a new, large-scale dataset for the advancement of change detection and building damage assessment for humanitarian assistance and disaster recovery research. Natural disaster response requires an accurate understanding of damaged buildings in an affected region. Current response strategies require in-person damage assessments within 24-48 hours of a disaster. Massive potential exists for using aerial imagery combined with computer vision algorithms to assess damage and reduce the potential danger to human life. In collaboration with multiple disaster response agencies, xBD provides pre- and post-event satellite imagery across a variety of disaster events with building polygons, ordinal labels of damage level, and corresponding satellite metadata. Furthermore, the dataset contains bounding boxes and labels for environmental factors such as fire, water, and smoke. xBD is the largest building damage assessment dataset to date, containing 850,736 building annotations across 45,362 km\textsuperscript{2} of imagery.
△ Less
Submitted 21 November, 2019;
originally announced November 2019.
-
A Smart Wizard System Suitable for Use With Internet Mobile Devices to Adjust Personal Information Privacy Settings
Authors:
Nahier Aldhafferi,
Charles Watson,
A. S. M Sajeev
Abstract:
The privacy of personal information is an important issue affecting the confidence of internet users. The widespread adoption of online social networks and access to these platforms using mobile devices has encouraged developers to make the systems and interfaces acceptable to users who seek privacy. The aim of this study is to test a wizard that allows users to control the sharing of personal inf…
▽ More
The privacy of personal information is an important issue affecting the confidence of internet users. The widespread adoption of online social networks and access to these platforms using mobile devices has encouraged developers to make the systems and interfaces acceptable to users who seek privacy. The aim of this study is to test a wizard that allows users to control the sharing of personal information with others. We also assess the concerns of users in terms of such sharing such as whether to hide personal data in current online social network accounts. Survey results showed the wizard worked very well and that females concealed more personal information than did males. In addition, most users who were concerned about misuse of personal information hid those items. The results can be used to upgrade current privacy systems or to design new systems that work on mobile internet devices. The system can also be used to save time when setting personal privacy settings and makes users more aware of items that will be shared with others.
△ Less
Submitted 13 July, 2013;
originally announced July 2013.
-
Personal Information Privacy Settings of Online Social Networks and their Suitability for Mobile Internet Devices
Authors:
Nahier Aldhafferi,
Charles Watson,
A. S. M Sajeev
Abstract:
Protecting personal information privacy has become a controversial issue among online social network providers and users. Most social network providers have developed several techniques to decrease threats and risks to the users privacy. These risks include the misuse of personal information which may lead to illegal acts such as identity theft. This study aims to measure the awareness of users on…
▽ More
Protecting personal information privacy has become a controversial issue among online social network providers and users. Most social network providers have developed several techniques to decrease threats and risks to the users privacy. These risks include the misuse of personal information which may lead to illegal acts such as identity theft. This study aims to measure the awareness of users on protecting their personal information privacy, as well as the suitability of the privacy systems which they use to modify privacy settings. Survey results show high percentage of the use of smart phones for web services but the current privacy settings for online social networks need to be improved to support different type of mobile phones screens. Because most users use their mobile phones for Internet services, privacy settings that are compatible with mobile phones need to be developed. The method of selecting privacy settings should also be simplified to provide users with a clear picture of the data that will be shared with others. Results of this study can be used to develop a new privacy system which will help users control their personal information easily from different devices, including mobile Internet devices and computers.
△ Less
Submitted 13 May, 2013;
originally announced May 2013.
