-
Mitigating Exaggerated Safety in Large Language Models
Authors:
Ruchi Bhalani,
Ruchira Ray
Abstract:
As the popularity of Large Language Models (LLMs) grow, combining model safety with utility becomes increasingly important. The challenge is making sure that LLMs can recognize and decline dangerous prompts without sacrificing their ability to be helpful. The problem of "exaggerated safety" demonstrates how difficult this can be. To reduce excessive safety behaviours -- which was discovered to be…
▽ More
As the popularity of Large Language Models (LLMs) grow, combining model safety with utility becomes increasingly important. The challenge is making sure that LLMs can recognize and decline dangerous prompts without sacrificing their ability to be helpful. The problem of "exaggerated safety" demonstrates how difficult this can be. To reduce excessive safety behaviours -- which was discovered to be 26.1% of safe prompts being misclassified as dangerous and refused -- we use a combination of XSTest dataset prompts as well as interactive, contextual, and few-shot prompting to examine the decision bounds of LLMs such as Llama2, Gemma Command R+, and Phi-3. We find that few-shot prompting works best for Llama2, interactive prompting works best Gemma, and contextual prompting works best for Command R+ and Phi-3. Using a combination of these prompting strategies, we are able to mitigate exaggerated safety behaviors by an overall 92.9% across all LLMs. Our work presents a multiple prompting strategies to jailbreak LLMs' decision-making processes, allowing them to navigate the tight line between refusing unsafe prompts and remaining helpful.
△ Less
Submitted 8 May, 2024;
originally announced May 2024.
-
Controlled Training Data Generation with Diffusion Models
Authors:
Teresa Yeo,
Andrei Atanov,
Harold Benoit,
Aleksandr Alekseev,
Ruchira Ray,
Pooya Esmaeil Akhoondi,
Amir Zamir
Abstract:
In this work, we present a method to control a text-to-image generative model to produce training data specifically "useful" for supervised learning. Unlike previous works that employ an open-loop approach and pre-define prompts to generate new data using either a language model or human expertise, we develop an automated closed-loop system which involves two feedback mechanisms. The first mechani…
▽ More
In this work, we present a method to control a text-to-image generative model to produce training data specifically "useful" for supervised learning. Unlike previous works that employ an open-loop approach and pre-define prompts to generate new data using either a language model or human expertise, we develop an automated closed-loop system which involves two feedback mechanisms. The first mechanism uses feedback from a given supervised model and finds adversarial prompts that result in image generations that maximize the model loss. While these adversarial prompts result in diverse data informed by the model, they are not informed of the target distribution, which can be inefficient. Therefore, we introduce the second feedback mechanism that guides the generation process towards a certain target distribution. We call the method combining these two mechanisms Guided Adversarial Prompts. We perform our evaluations on different tasks, datasets and architectures, with different types of distribution shifts (spuriously correlated data, unseen domains) and demonstrate the efficiency of the proposed feedback mechanisms compared to open-loop approaches.
△ Less
Submitted 22 March, 2024;
originally announced March 2024.
-
Queer In AI: A Case Study in Community-Led Participatory AI
Authors:
Organizers Of QueerInAI,
:,
Anaelia Ovalle,
Arjun Subramonian,
Ashwin Singh,
Claas Voelcker,
Danica J. Sutherland,
Davide Locatelli,
Eva Breznik,
Filip Klubička,
Hang Yuan,
Hetvi J,
Huan Zhang,
Jaidev Shriram,
Kruno Lehman,
Luca Soldaini,
Maarten Sap,
Marc Peter Deisenroth,
Maria Leonor Pacheco,
Maria Ryskina,
Martin Mundt,
Milind Agarwal,
Nyx McLean,
Pan Xu,
A Pranav
, et al. (26 additional authors not shown)
Abstract:
We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over the years. We discuss different challenges that emerged in the process, look at ways this organization has fallen short of operationalizing participatory and intersectional principles, and then assess th…
▽ More
We present Queer in AI as a case study for community-led participatory design in AI. We examine how participatory design and intersectional tenets started and shaped this community's programs over the years. We discuss different challenges that emerged in the process, look at ways this organization has fallen short of operationalizing participatory and intersectional principles, and then assess the organization's impact. Queer in AI provides important lessons and insights for practitioners and theorists of participatory methods broadly through its rejection of hierarchy in favor of decentralization, success at building aid and programs by and for the queer community, and effort to change actors and institutions outside of the queer community. Finally, we theorize how communities like Queer in AI contribute to the participatory design in AI more broadly by fostering cultures of participation in AI, welcoming and empowering marginalized participants, critiquing poor or exploitative participatory practices, and bringing participation to institutions outside of individual research projects. Queer in AI's work serves as a case study of grassroots activism and participatory methods within AI, demonstrating the potential of community-led participatory methods and intersectional praxis, while also providing challenges, case studies, and nuanced insights to researchers develo** and using participatory methods.
△ Less
Submitted 8 June, 2023; v1 submitted 29 March, 2023;
originally announced March 2023.
-
A Comparison of Speech Data Augmentation Methods Using S3PRL Toolkit
Authors:
Mina Huh,
Ruchira Ray,
Corey Karnei
Abstract:
Data augmentations are known to improve robustness in speech-processing tasks. In this study, we summarize and compare different data augmentation strategies using S3PRL toolkit. We explore how HuBERT and wav2vec perform using different augmentation techniques (SpecAugment, Gaussian Noise, Speed Perturbation) for Phoneme Recognition (PR) and Automatic Speech Recognition (ASR) tasks. We evaluate mo…
▽ More
Data augmentations are known to improve robustness in speech-processing tasks. In this study, we summarize and compare different data augmentation strategies using S3PRL toolkit. We explore how HuBERT and wav2vec perform using different augmentation techniques (SpecAugment, Gaussian Noise, Speed Perturbation) for Phoneme Recognition (PR) and Automatic Speech Recognition (ASR) tasks. We evaluate model performance in terms of phoneme error rate (PER) and word error rate (WER). From the experiments, we observed that SpecAugment slightly improves the performance of HuBERT and wav2vec on the original dataset. Also, we show that models trained using the Gaussian Noise and Speed Perturbation dataset are more robust when tested with augmented test sets.
△ Less
Submitted 29 March, 2024; v1 submitted 27 February, 2023;
originally announced March 2023.
-
Fast Falsification of Neural Networks using Property Directed Testing
Authors:
Moumita Das,
Rajarshi Ray,
Swarup Kumar Mohalik,
Ansuman Banerjee
Abstract:
Neural networks are now extensively used in perception, prediction and control of autonomous systems. Their deployment in safety-critical systems brings forth the need for verification techniques for such networks. As an alternative to exhaustive and costly verification algorithms, lightweight falsification algorithms have been heavily used to search for an input to the system that produces an uns…
▽ More
Neural networks are now extensively used in perception, prediction and control of autonomous systems. Their deployment in safety-critical systems brings forth the need for verification techniques for such networks. As an alternative to exhaustive and costly verification algorithms, lightweight falsification algorithms have been heavily used to search for an input to the system that produces an unsafe output, i.e., a counterexample to the safety of the system. In this work, we propose a falsification algorithm for neural networks that directs the search for a counterexample, guided by a safety property specification. Our algorithm uses a derivative-free sampling-based optimization method. We evaluate our algorithm on 45 trained neural network benchmarks of the ACAS Xu system against 10 safety properties. We show that our falsification procedure detects all the unsafe instances that other verification tools also report as unsafe. Moreover, in terms of performance, our falsification procedure identifies most of the unsafe instances faster, in comparison to the state-of-the-art verification tools for feed-forward neural networks such as NNENUM and Neurify and in many instances, by orders of magnitude.
△ Less
Submitted 26 April, 2021;
originally announced April 2021.
-
Modular and fractional L-intersecting families of vector spaces
Authors:
Rogers Mathew,
Tapas Kumar Mishra,
Ritabrata Ray,
Shashank Srivastava
Abstract:
In the first part of this paper, we prove a theorem which is the $q$-analogue of a generalized modular Ray-Chaudhuri-Wilson Theorem shown in [Alon, Babai, Suzuki, J. Combin. Theory Series A, 1991]. It is also a generalization of the main theorem in [Frankl and Graham, European J. Combin. 1985] under certain circumstances.
In the second part of this paper, we prove $q$-analogues of results on a r…
▽ More
In the first part of this paper, we prove a theorem which is the $q$-analogue of a generalized modular Ray-Chaudhuri-Wilson Theorem shown in [Alon, Babai, Suzuki, J. Combin. Theory Series A, 1991]. It is also a generalization of the main theorem in [Frankl and Graham, European J. Combin. 1985] under certain circumstances.
In the second part of this paper, we prove $q$-analogues of results on a recent notion called \emph{fractional $L$-intersecting family} for families of subspaces of a given vector space. We use the above theorem to obtain a general upper bound to the cardinality of such families. We give an improvement to this general upper bound in certain special cases.
△ Less
Submitted 4 June, 2020; v1 submitted 10 April, 2020;
originally announced April 2020.
-
Fractional cross intersecting families
Authors:
Rogers Mathew,
Ritabrata Ray,
Shashank Srivastava
Abstract:
Let $\mathcal{A}=\{A_{1},...,A_{p}\}$ and $\mathcal{B}=\{B_{1},...,B_{q}\}$ be two families of subsets of $[n]$ such that for every $i\in [p]$ and $j\in [q]$, $|A_{i}\cap B_{j}|= \frac{c}{d}|B_{j}|$, where $\frac{c}{d}\in [0,1]$ is an irreducible fraction. We call such families "$\frac{c}{d}$-cross intersecting families". In this paper, we find a tight upper bound for the product…
▽ More
Let $\mathcal{A}=\{A_{1},...,A_{p}\}$ and $\mathcal{B}=\{B_{1},...,B_{q}\}$ be two families of subsets of $[n]$ such that for every $i\in [p]$ and $j\in [q]$, $|A_{i}\cap B_{j}|= \frac{c}{d}|B_{j}|$, where $\frac{c}{d}\in [0,1]$ is an irreducible fraction. We call such families "$\frac{c}{d}$-cross intersecting families". In this paper, we find a tight upper bound for the product $|\mathcal{A}||\mathcal{B}|$ and characterize the cases when this bound is achieved for $\frac{c}{d}=\frac{1}{2}$. Also, we find a tight upper bound on $|\mathcal{A}||\mathcal{B}|$ when $\mathcal{B}$ is $k$-uniform and characterize, for all $\frac{c}{d}$, the cases when this bound is achieved.
△ Less
Submitted 5 March, 2019;
originally announced March 2019.
-
Simultaneous Solving of Batched Linear Programs on a GPU
Authors:
Amit Gurung,
Rajarshi Ray
Abstract:
Linear Programs (LPs) appear in a large number of applications and offloading them to a GPU is viable to gain performance. Existing work on offloading and solving an LP on a GPU suggests that there is performance gain generally on large sized LPs (typically 500 constraints, 500 variables and above). In order to gain performance from a GPU, for applications involving small to medium sized LPs, we p…
▽ More
Linear Programs (LPs) appear in a large number of applications and offloading them to a GPU is viable to gain performance. Existing work on offloading and solving an LP on a GPU suggests that there is performance gain generally on large sized LPs (typically 500 constraints, 500 variables and above). In order to gain performance from a GPU, for applications involving small to medium sized LPs, we propose batched solving of a large number of LPs in parallel. In this paper, we present the design and implementation of a batched LP solver in CUDA, kee** memory coalescent access, low CPU-GPU memory transfer latency and load balancing as the goals. The performance of the batched LP solver is compared against sequential solving in the CPU using the open source solver GLPK (GNU Linear Programming Kit) and the CPLEX solver from IBM. The evaluation on selected LP benchmarks from the Netlib repository displays a maximum speed-up of 95x and 5x with respect to CPLEX and GLPK solver respectively, for a batch of 1e5 LPs. We demonstrate the application of our batched LP solver to enhance performance in the domain of state-space exploration of mathematical models of control systems design.
△ Less
Submitted 21 February, 2018;
originally announced February 2018.
-
Exact Synthesis of Reversible Logic Circuits using Model Checking
Authors:
Rajarshi Ray,
Arup Deka,
Kamalika Datta
Abstract:
Synthesis of reversible logic circuits has gained great atten- tion during the last decade. Various synthesis techniques have been pro- posed, some generate optimal solutions (in gate count) and are termed as exact, while others are scalable in the sense that they can handle larger functions but generate sub-optimal solutions. Although scalable synthe- sis is very much essential for circuit design…
▽ More
Synthesis of reversible logic circuits has gained great atten- tion during the last decade. Various synthesis techniques have been pro- posed, some generate optimal solutions (in gate count) and are termed as exact, while others are scalable in the sense that they can handle larger functions but generate sub-optimal solutions. Although scalable synthe- sis is very much essential for circuit design, exact synthesis is also of great importance as it helps in building design library for the synthesis of larger functions. In this paper, we propose an exact synthesis technique for re- versible circuits using model checking. We frame the synthesis problem as a model checking instance and propose an iterative bounded model checking calls for an optimal synthesis. Experiments on reversible logic benchmarks shows successful synthesis of optimal circuits. We also illus- trate optimal synthesis of random functions with as many as 10 variables and up to 10 gates.
△ Less
Submitted 24 February, 2017;
originally announced February 2017.
-
An Efficient Algorithm for Vertex Enumeration of Two-Dimensional Projection of Polytopes
Authors:
Amit Gurung,
Rajarshi Ray
Abstract:
An efficient algorithm to enumerate the vertices of a two-dimensional (2D) projection of a polytope, is presented in this paper. The proposed algorithm uses the support function of the polytope to be projected and enumerated for vertices. The complexity of our algorithm is linear in the number of vertices of the projected polytope and we show empirically that the performance is significantly bette…
▽ More
An efficient algorithm to enumerate the vertices of a two-dimensional (2D) projection of a polytope, is presented in this paper. The proposed algorithm uses the support function of the polytope to be projected and enumerated for vertices. The complexity of our algorithm is linear in the number of vertices of the projected polytope and we show empirically that the performance is significantly better in comparison to some known efficient algorithms of projection and enumeration.
△ Less
Submitted 30 November, 2016;
originally announced November 2016.
-
Solving Batched Linear Programs on GPU and Multicore CPU
Authors:
Amit Gurung,
Rajarshi Ray
Abstract:
Linear Programs (LPs) appear in a large number of applications and offloading them to the GPU is viable to gain performance. Existing work on offloading and solving an LP on GPU suggests that performance is gained from large sized LPs (typically 500 constraints, 500 variables and above). In order to gain performance from GPU for applications involving small to medium sized LPs, we propose batched…
▽ More
Linear Programs (LPs) appear in a large number of applications and offloading them to the GPU is viable to gain performance. Existing work on offloading and solving an LP on GPU suggests that performance is gained from large sized LPs (typically 500 constraints, 500 variables and above). In order to gain performance from GPU for applications involving small to medium sized LPs, we propose batched solving of a large number of LPs in parallel. In this paper, we present the design and CUDA implementation of our batched LP solver library, kee** memory coalescent access, reduced CPU-GPU memory transfer latency and load balancing as the goals. The performance of the batched LP solver is compared against sequential solving in the CPU using an open source solver GLPK (GNU Linear Programming Kit). The performance is evaluated for three types of LPs. The first type is the initial basic solution as feasible, the second type is the initial basic solution as infeasible and the third type is the feasible region as a Hyperbox. For the first type, we show a maximum speedup of $18.3\times$ when running a batch of $50k$ LPs of size $100$ ($100$ variables, $100$ constraints). For the second type, a maximum speedup of $12\times$ is obtained with a batch of $10k$ LPs of size $200$. For the third type, we show a significant speedup of $63\times$ in solving a batch of nearly $4$ million LPs of size 5 and $34\times$ in solving 6 million LPs of size $28$. In addition, we show that the open source library for solving linear programs-GLPK, can be easily extended to solve many LPs in parallel with multi-threading. The thread parallel GLPK implementation runs $9.6\times$ faster in solving a batch of $1e5$ LPs of size $100$, on a $12$-core Intel Xeon processor. We demonstrate the application of our batched LP solver in the domain of state-space exploration of mathematical models of control systems design.
△ Less
Submitted 26 September, 2016;
originally announced September 2016.
-
Parallel Reachability Analysis for Hybrid Systems
Authors:
Amit Gurung,
Arup Deka,
Ezio Bartocci,
Sergiy Bogomolov,
Radu Grosu,
Rajarshi Ray
Abstract:
We propose two parallel state-space exploration algorithms for hybrid systems with the goal of enhancing performance on multi-core shared memory systems. The first is an adaption of the parallel breadth first search in the SPIN model checker. We show that the adapted algorithm does not provide the desired load balancing for many hybrid systems benchmarks. The second is a task parallel algorithm ba…
▽ More
We propose two parallel state-space exploration algorithms for hybrid systems with the goal of enhancing performance on multi-core shared memory systems. The first is an adaption of the parallel breadth first search in the SPIN model checker. We show that the adapted algorithm does not provide the desired load balancing for many hybrid systems benchmarks. The second is a task parallel algorithm based on cheaply precomputing cost of post (continuous and discrete) operations for effective load balancing. We illustrate the task parallel algorithm and the cost precomputation of post operators on a support-function-based algorithm for state-space exploration. The performance comparison of the two algorithms displays a better CPU utilization/load-balancing of the second over the first, except for certain cases. The algorithms are implemented in the model checker XSpeed and our experiments show a maximum speed-up of $900\times$ on a navigation benchmark with respect to SpaceEx LGG scenario, comparing on the basis of equal number of post operations evaluated.
△ Less
Submitted 17 June, 2016;
originally announced June 2016.
-
Towards Refactoring DMARF and GIPSY OSS
Authors:
Aaradhna Goyal,
Ali Alshamrani,
Dhivyaa Nandakumar,
Dileep Vanga,
Dmitriy Fingerman,
Parul Gupta,
Riya Ray,
Srikanth Suryadevara
Abstract:
We present here an exploratory and investigatory study of the requirements, design, and implementation of two opensource software systems: the Distributed Modular Audio Recognition Framework (DMARF), and the General Intensional Programming System (GIPSY). The inception, development, and evolution of the two systems have overlapped and in terms of the involved developers, as well as in their applic…
▽ More
We present here an exploratory and investigatory study of the requirements, design, and implementation of two opensource software systems: the Distributed Modular Audio Recognition Framework (DMARF), and the General Intensional Programming System (GIPSY). The inception, development, and evolution of the two systems have overlapped and in terms of the involved developers, as well as in their applications. DMARF is a platform independent collection of algorithms for pattern recognition, identification and signal processing in audio and natural language text samples, become a rich platform for the research community in particular to use, test, and compare various algorithms in the broad field of pattern recognition and machine learning. Intended as a platform for intensional programming, GIPSY's inception was intended to push the field of intensional programming further, overcoming limitations in the available tools two decades ago. In this study, we present background research into the two systems and elaborate on their motivations and the requirements that drove and shaped their design and implementation. We subsequently elaborate in more depth about various aspects their architectural design, including the elucidation of some use cases, domain models, and the overall class diagram of the major components. Moreover, we investigated existing design patterns in both systems and provided a detailed view of the involved components in such patterns. Furthermore, we delve deeper into the guts of both systems, identifying code smells and suggesting possible refactorings. Patchsets of implementations of selected refactorings have been collected into patchsets and could be committed into future releases of the two systems, pending a review and approval of the developers and maintainers of DMARF and GIPSY.
△ Less
Submitted 25 November, 2014; v1 submitted 14 October, 2014;
originally announced October 2014.
-
Autonomous Navigation by Robust Scan Matching Technique
Authors:
Debajyoti Banerji,
Ranjit Ray,
Jhankar Basu,
Indrajit Basak
Abstract:
For effective autonomous navigation,estimation of the pose of the robot is essential at every sampling time. For computing an accurate estimation,odometric error needs to be reduced with the help of data from external sensor. In this work, a technique has been developed for accurate pose estimation of mobile robot by using Laser Range data. The technique is robust to noisy data, which may contain…
▽ More
For effective autonomous navigation,estimation of the pose of the robot is essential at every sampling time. For computing an accurate estimation,odometric error needs to be reduced with the help of data from external sensor. In this work, a technique has been developed for accurate pose estimation of mobile robot by using Laser Range data. The technique is robust to noisy data, which may contain considerable amount of outliers. A grey image is formed from laser range data and the key points from this image are extracted by Harris corner detector. The matching of the key points from consecutive data sets have been done while outliers have been rejected by RANSAC method. Robot state is measured by the correspondence between the two sets of keypoints. Finally, optimal robot state is estimated by Extended Kalman Filter. The technique has been applied to an operational robot in the laboratory environment to show the robustness of the technique in presence of noisy sensor data. The performance of this new technique has been compared with that of conventional ICP method. Through this method, effective and accurate navigation has been achieved even in presence of substantial noise in the sensor data at the cost of a small amount of additional computational complexity.
△ Less
Submitted 6 December, 2012;
originally announced December 2012.
-
A Cost Effective RFID Based Customized DVD-ROM to Thwart Software Piracy
Authors:
Sudip Dogra,
Ritwik Ray,
Saustav Ghosh,
Debharshi Bhattacharya,
Subir Kr. Sarkar
Abstract:
Software piracy has been a very perilous adversary of the software based industry, from the very beginning of the development of the latter into a significant business. There has been no developed foolproof system that has been developed to appropriately tackle this vile issue. We have in our scheme tried to develop a way to embark upon this problem using a very recently developed technology of…
▽ More
Software piracy has been a very perilous adversary of the software based industry, from the very beginning of the development of the latter into a significant business. There has been no developed foolproof system that has been developed to appropriately tackle this vile issue. We have in our scheme tried to develop a way to embark upon this problem using a very recently developed technology of RFID.
△ Less
Submitted 2 November, 2009;
originally announced November 2009.
