-
Safe and Responsible Large Language Model : Can We Balance Bias Reduction and Language Understanding in Large Language Models?
Authors:
Shaina Raza,
Oluwanifemi Bamgbose,
Shardul Ghuge,
Fatemeh Tavakol,
Deepak John Reji,
Syed Raza Bashir
Abstract:
Large Language Models (LLMs) have significantly advanced various NLP tasks. However, these models often risk generating unsafe text that perpetuates biases. Current approaches to produce unbiased outputs from LLMs can reduce biases but at the expense of knowledge retention. In this research, we address the question of whether producing safe (unbiased) outputs through LLMs can retain knowledge and…
▽ More
Large Language Models (LLMs) have significantly advanced various NLP tasks. However, these models often risk generating unsafe text that perpetuates biases. Current approaches to produce unbiased outputs from LLMs can reduce biases but at the expense of knowledge retention. In this research, we address the question of whether producing safe (unbiased) outputs through LLMs can retain knowledge and language understanding. In response, we developed the Safety and Responsible Large Language Model (\textbf{SR}$_{\text{LLM}}$), an LLM that has been instruction fine-tuned on top of already safe LLMs (e.g., Llama2 or related) to diminish biases in generated text. To achieve our goals, we compiled a specialized dataset designed to train our model in identifying and correcting biased text. We conduct experiments, both on this custom data and out-of-distribution test sets, to show the bias reduction and knowledge retention. The results confirm that \textbf{SR}$_{\text{LLM}}$ outperforms traditional fine-tuning and prompting methods in both reducing biases and preserving the integrity of language knowledge. The significance of our findings lies in demonstrating that instruction fine-tuning can provide a more robust solution for bias reduction in LLMs. We have made our code and data available at \href{https://github.com/shainarazavi/Safe-Responsible-LLM}{Safe-LLM}.
△ Less
Submitted 1 July, 2024; v1 submitted 1 April, 2024;
originally announced April 2024.
-
Empowering Communication: Speech Technology for Indian and Western Accents through AI-powered Speech Synthesis
Authors:
Vinotha R,
Hepsiba D,
L. D. Vijay Anand,
Deepak John Reji
Abstract:
Neural Text-to-speech (TTS) synthesis is a powerful technology that can generate speech using neural networks. One of the most remarkable features of TTS synthesis is its capability to produce speech in the voice of different speakers. This paper introduces voice cloning and speech synthesis https://pypi.org/project/voice-cloning/ an open-source python package for hel** speech disorders to commu…
▽ More
Neural Text-to-speech (TTS) synthesis is a powerful technology that can generate speech using neural networks. One of the most remarkable features of TTS synthesis is its capability to produce speech in the voice of different speakers. This paper introduces voice cloning and speech synthesis https://pypi.org/project/voice-cloning/ an open-source python package for hel** speech disorders to communicate more effectively as well as for professionals seeking to integrate voice cloning or speech synthesis capabilities into their projects. This package aims to generate synthetic speech that sounds like the natural voice of an individual, but it does not replace the natural human voice. The architecture of the system comprises a speaker verification system, a synthesizer, a vocoder, and noise reduction. Speaker verification system trained on a varied set of speakers to achieve optimal generalization performance without relying on transcriptions. Synthesizer is trained using both audio and transcriptions that generate Mel spectrogram from a text and vocoder which converts the generated Mel Spectrogram into corresponding audio signal. Then the audio signal is processed by a noise reduction algorithm to eliminate unwanted noise and enhance speech clarity. The performance of synthesized speech from seen and unseen speakers are then evaluated using subjective and objective evaluation such as Mean Opinion Score (MOS), Gross Pitch Error (GPE), and Spectral distortion (SD). The model can create speech in distinct voices by including speaker characteristics that are chosen randomly.
△ Less
Submitted 16 February, 2024; v1 submitted 22 January, 2024;
originally announced January 2024.
-
NBIAS: A Natural Language Processing Framework for Bias Identification in Text
Authors:
Shaina Raza,
Muskan Garg,
Deepak John Reji,
Syed Raza Bashir,
Chen Ding
Abstract:
Bias in textual data can lead to skewed interpretations and outcomes when the data is used. These biases could perpetuate stereotypes, discrimination, or other forms of unfair treatment. An algorithm trained on biased data may end up making decisions that disproportionately impact a certain group of people. Therefore, it is crucial to detect and remove these biases to ensure the fair and ethical u…
▽ More
Bias in textual data can lead to skewed interpretations and outcomes when the data is used. These biases could perpetuate stereotypes, discrimination, or other forms of unfair treatment. An algorithm trained on biased data may end up making decisions that disproportionately impact a certain group of people. Therefore, it is crucial to detect and remove these biases to ensure the fair and ethical use of data. To this end, we develop a comprehensive and robust framework NBIAS that consists of four main layers: data, corpus construction, model development and an evaluation layer. The dataset is constructed by collecting diverse data from various domains, including social media, healthcare, and job hiring portals. As such, we applied a transformer-based token classification model that is able to identify bias words/ phrases through a unique named entity BIAS. In the evaluation procedure, we incorporate a blend of quantitative and qualitative measures to gauge the effectiveness of our models. We achieve accuracy improvements ranging from 1% to 8% compared to baselines. We are also able to generate a robust understanding of the model functioning. The proposed approach is applicable to a variety of biases and contributes to the fair and ethical use of textual data.
△ Less
Submitted 29 August, 2023; v1 submitted 3 August, 2023;
originally announced August 2023.
-
Dbias: Detecting biases and ensuring Fairness in news articles
Authors:
Shaina Raza,
Deepak John Reji,
Chen Ding
Abstract:
Because of the increasing use of data-centric systems and algorithms in machine learning, the topic of fairness is receiving a lot of attention in the academic and broader literature. This paper introduces Dbias (https://pypi.org/project/Dbias/), an open-source Python package for ensuring fairness in news articles. Dbias can take any text to determine if it is biased. Then, it detects biased words…
▽ More
Because of the increasing use of data-centric systems and algorithms in machine learning, the topic of fairness is receiving a lot of attention in the academic and broader literature. This paper introduces Dbias (https://pypi.org/project/Dbias/), an open-source Python package for ensuring fairness in news articles. Dbias can take any text to determine if it is biased. Then, it detects biased words in the text, masks them, and suggests a set of sentences with new words that are bias-free or at least less biased. We conduct extensive experiments to assess the performance of Dbias. To see how well our approach works, we compare it to the existing fairness models. We also test the individual components of Dbias to see how effective they are. The experimental results show that Dbias outperforms all the baselines in terms of accuracy and fairness. We make this package (Dbias) as publicly available for the developers and practitioners to mitigate biases in textual data (such as news articles), as well as to encourage extension of this work.
△ Less
Submitted 11 August, 2022;
originally announced August 2022.
-
An Approach to Ensure Fairness in News Articles
Authors:
Shaina Raza,
Deepak John Reji,
Dora D. Liu,
Syed Raza Bashir,
Usman Naseem
Abstract:
Recommender systems, information retrieval, and other information access systems present unique challenges for examining and applying concepts of fairness and bias mitigation in unstructured text. This paper introduces Dbias, which is a Python package to ensure fairness in news articles. Dbias is a trained Machine Learning (ML) pipeline that can take a text (e.g., a paragraph or news story) and de…
▽ More
Recommender systems, information retrieval, and other information access systems present unique challenges for examining and applying concepts of fairness and bias mitigation in unstructured text. This paper introduces Dbias, which is a Python package to ensure fairness in news articles. Dbias is a trained Machine Learning (ML) pipeline that can take a text (e.g., a paragraph or news story) and detects if the text is biased or not. Then, it detects the biased words in the text, masks them, and recommends a set of sentences with new words that are bias-free or at least less biased. We incorporate the elements of data science best practices to ensure that this pipeline is reproducible and usable. We show in experiments that this pipeline can be effective for mitigating biases and outperforms the common neural network architectures in ensuring fairness in the news articles.
△ Less
Submitted 8 July, 2022;
originally announced July 2022.
