-
Large Language Models have Intrinsic Self-Correction Ability
Authors:
Dancheng Liu,
Amir Nassereldine,
Ziming Yang,
Chenhui Xu,
Yuting Hu,
Jiajie Li,
Utkarsh Kumar,
Changjae Lee,
**jun Xiong
Abstract:
Large language models (LLMs) have attracted significant attention for their remarkable abilities in various natural language processing tasks, but they suffer from hallucinations that will cause performance degradation. One promising solution to improve the LLMs' performance is to ask LLMs to revise their answer after generation, a technique known as self-correction. Among the two types of self-co…
▽ More
Large language models (LLMs) have attracted significant attention for their remarkable abilities in various natural language processing tasks, but they suffer from hallucinations that will cause performance degradation. One promising solution to improve the LLMs' performance is to ask LLMs to revise their answer after generation, a technique known as self-correction. Among the two types of self-correction, intrinsic self-correction is considered a promising direction because it does not utilize external knowledge. However, recent works doubt the validity of LLM's ability to conduct intrinsic self-correction. In this paper, we present a novel perspective on the intrinsic self-correction capabilities of LLMs through theoretical analyses and empirical experiments. In addition, we identify two critical factors for successful self-correction: zero temperature and fair prompts. Leveraging these factors, we demonstrate that intrinsic self-correction ability is exhibited across multiple existing LLMs. Our findings offer insights into the fundamental theories underlying the self-correction behavior of LLMs and remark on the importance of unbiased prompts and zero temperature settings in harnessing their full potential.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
PI-Whisper: An Adaptive and Incremental ASR Framework for Diverse and Evolving Speaker Characteristics
Authors:
Amir Nassereldine,
Dancheng Liu,
Chenhui Xu,
**jun Xiong
Abstract:
As edge-based automatic speech recognition (ASR) technologies become increasingly prevalent for the development of intelligent and personalized assistants, three important challenges must be addressed for these resource-constrained ASR models, i.e., adaptivity, incrementality, and inclusivity. We propose a novel ASR framework, PI-Whisper, in this work and show how it can improve an ASR's recogniti…
▽ More
As edge-based automatic speech recognition (ASR) technologies become increasingly prevalent for the development of intelligent and personalized assistants, three important challenges must be addressed for these resource-constrained ASR models, i.e., adaptivity, incrementality, and inclusivity. We propose a novel ASR framework, PI-Whisper, in this work and show how it can improve an ASR's recognition capabilities adaptively by identifying different speakers' characteristics in real-time, how such an adaption can be performed incrementally without repetitive retraining, and how it can improve the equity and fairness for diverse speaker groups. More impressively, our proposed PI-Whisper framework attains all of these nice properties while still achieving state-of-the-art accuracy with up to 13.7% reduction of the word error rate (WER) with linear scalability with respect to computing resources.
△ Less
Submitted 21 June, 2024;
originally announced June 2024.
-
Predicting the Performance-Cost Trade-off of Applications Across Multiple Systems
Authors:
Amir Nassereldine,
Safaa Diab,
Mohammed Baydoun,
Kenneth Leach,
Maxim Alt,
Dejan Milojicic,
Izzat El Hajj
Abstract:
In modern computing environments, users may have multiple systems accessible to them such as local clusters, private clouds, or public clouds. This abundance of choices makes it difficult for users to select the system and configuration for running an application that best meet their performance and cost objectives. To assist such users, we propose a prediction tool that predicts the full performa…
▽ More
In modern computing environments, users may have multiple systems accessible to them such as local clusters, private clouds, or public clouds. This abundance of choices makes it difficult for users to select the system and configuration for running an application that best meet their performance and cost objectives. To assist such users, we propose a prediction tool that predicts the full performance-cost trade-off space of an application across multiple systems. Our tool runs and profiles a submitted application on a small number of configurations from some of the systems, and uses that information to predict the application's performance on all configurations in all systems. The prediction models are trained offline with data collected from running a large number of applications on a wide variety of configurations. Notable aspects of our tool include: providing different scopes of prediction with varying online profiling requirements, automating the selection of the small number of configurations and systems used for online profiling, performing online profiling using partial runs thereby make predictions for applications without running them to completion, employing a classifier to distinguish applications that scale well from those that scale poorly, and predicting the sensitivity of applications to interference from other users. We evaluate our tool using 69 data analytics and scientific computing benchmarks executing on three different single-node CPU systems with 8-9 configurations each and show that it can achieve low prediction error with modest profiling overhead.
△ Less
Submitted 4 April, 2023;
originally announced April 2023.
-
Excel Spreadsheet Analyzer
Authors:
Amir Nassereldine,
Patrick Chen,
**jun Xiong
Abstract:
Spreadsheets are widely used in various fields to do large numerical analysis. While several companies have relied on spreadsheets for decades, data scientists are going in the direction of using scientific programming languages such as python to do their data analysis due to the support, community, and vast amount of libraries. While using python to analyze a company's spreadsheets, some informat…
▽ More
Spreadsheets are widely used in various fields to do large numerical analysis. While several companies have relied on spreadsheets for decades, data scientists are going in the direction of using scientific programming languages such as python to do their data analysis due to the support, community, and vast amount of libraries. While using python to analyze a company's spreadsheets, some information such as the formulas and dependencies of a cell are lost. We propose a tool that creates an abstract intermediate representation (AIR) of a spreadsheet. This representation facilitates the transfer from spreadsheets into scientific programming languages while preserving inter-dependency information about data. In addition to that, we build a python library on top of our tool to perform some data analysis in python.
△ Less
Submitted 1 November, 2022;
originally announced November 2022.
-
A Framework for High-throughput Sequence Alignment using Real Processing-in-Memory Systems
Authors:
Safaa Diab,
Amir Nassereldine,
Mohammed Alser,
Juan Gómez-Luna,
Onur Mutlu,
Izzat El Hajj
Abstract:
Sequence alignment is a memory bound computation whose performance in modern systems is limited by the memory bandwidth bottleneck. Processing-in-memory architectures alleviate this bottleneck by providing the memory with computing competencies. We propose Alignment-in-Memory (AIM), a framework for high-throughput sequence alignment using processing-in-memory, and evaluate it on UPMEM, the first p…
▽ More
Sequence alignment is a memory bound computation whose performance in modern systems is limited by the memory bandwidth bottleneck. Processing-in-memory architectures alleviate this bottleneck by providing the memory with computing competencies. We propose Alignment-in-Memory (AIM), a framework for high-throughput sequence alignment using processing-in-memory, and evaluate it on UPMEM, the first publicly-available general-purpose programmable processing-in-memory system.
Our evaluation shows that a real processing-in-memory system can substantially outperform server-grade multi-threaded CPU systems running at full-scale when performing sequence alignment for a variety of algorithms, read lengths, and edit distance thresholds. We hope that our findings inspire more work on creating and accelerating bioinformatics algorithms for such real processing-in-memory systems.
Our code is available at https://github.com/safaad/aim.
△ Less
Submitted 27 March, 2023; v1 submitted 2 August, 2022;
originally announced August 2022.
-
High-throughput Pairwise Alignment with the Wavefront Algorithm using Processing-in-Memory
Authors:
Safaa Diab,
Amir Nassereldine,
Mohammed Alser,
Juan Gómez Luna,
Onur Mutlu,
Izzat El Hajj
Abstract:
We show that the wavefront algorithm can achieve higher pairwise read alignment throughput on a UPMEM PIM system than on a server-grade multi-threaded CPU system.
We show that the wavefront algorithm can achieve higher pairwise read alignment throughput on a UPMEM PIM system than on a server-grade multi-threaded CPU system.
△ Less
Submitted 23 April, 2022; v1 submitted 5 April, 2022;
originally announced April 2022.
