-
Towards a Robust Retrieval-Based Summarization System
Authors:
Shengjie Liu,
**g Wu,
**gyuan Bao,
Wenyi Wang,
Naira Hovakimyan,
Christopher G Healey
Abstract:
This paper describes an investigation of the robustness of large language models (LLMs) for retrieval augmented generation (RAG)-based summarization tasks. While LLMs provide summarization capabilities, their performance in complex, real-world scenarios remains under-explored. Our first contribution is LogicSumm, an innovative evaluation framework incorporating realistic scenarios to assess LLM ro…
▽ More
This paper describes an investigation of the robustness of large language models (LLMs) for retrieval augmented generation (RAG)-based summarization tasks. While LLMs provide summarization capabilities, their performance in complex, real-world scenarios remains under-explored. Our first contribution is LogicSumm, an innovative evaluation framework incorporating realistic scenarios to assess LLM robustness during RAG-based summarization. Based on limitations identified by LogiSumm, we then developed SummRAG, a comprehensive system to create training dialogues and fine-tune a model to enhance robustness within LogicSumm's scenarios. SummRAG is an example of our goal of defining structured methods to test the capabilities of an LLM, rather than addressing issues in a one-off fashion. Experimental results confirm the power of SummRAG, showcasing improved logical coherence and summarization quality. Data, corresponding model weights, and Python code are available online.
△ Less
Submitted 28 March, 2024;
originally announced March 2024.
-
Abstractive Summarization of Large Document Collections Using GPT
Authors:
Sengjie Liu,
Christopher G. Healey
Abstract:
This paper proposes a method of abstractive summarization designed to scale to document collections instead of individual documents. Our approach applies a combination of semantic clustering, document size reduction within topic clusters, semantic chunking of a cluster's documents, GPT-based summarization and concatenation, and a combined sentiment and text visualization of each topic to support e…
▽ More
This paper proposes a method of abstractive summarization designed to scale to document collections instead of individual documents. Our approach applies a combination of semantic clustering, document size reduction within topic clusters, semantic chunking of a cluster's documents, GPT-based summarization and concatenation, and a combined sentiment and text visualization of each topic to support exploratory data analysis. Statistical comparison of our results to existing state-of-the-art systems BART, BRIO, PEGASUS, and MoCa using ROGUE summary scores showed statistically equivalent performance with BART and PEGASUS on the CNN/Daily Mail test dataset, and with BART on the Gigaword test dataset. This finding is promising since we view document collection summarization as more challenging than individual document summarization. We conclude with a discussion of how issues of scale are
△ Less
Submitted 9 October, 2023;
originally announced October 2023.
-
High Resolution Face Completion with Multiple Controllable Attributes via Fully End-to-End Progressive Generative Adversarial Networks
Authors:
Zeyuan Chen,
Shaoliang Nie,
Tianfu Wu,
Christopher G. Healey
Abstract:
We present a deep learning approach for high resolution face completion with multiple controllable attributes (e.g., male and smiling) under arbitrary masks. Face completion entails understanding both structural meaningfulness and appearance consistency locally and globally to fill in "holes" whose content do not appear elsewhere in an input image. It is a challenging task with the difficulty leve…
▽ More
We present a deep learning approach for high resolution face completion with multiple controllable attributes (e.g., male and smiling) under arbitrary masks. Face completion entails understanding both structural meaningfulness and appearance consistency locally and globally to fill in "holes" whose content do not appear elsewhere in an input image. It is a challenging task with the difficulty level increasing significantly with respect to high resolution, the complexity of "holes" and the controllable attributes of filled-in fragments. Our system addresses the challenges by learning a fully end-to-end framework that trains generative adversarial networks (GANs) progressively from low resolution to high resolution with conditional vectors encoding controllable attributes.
We design novel network architectures to exploit information across multiple scales effectively and efficiently. We introduce new loss functions encouraging sharp completion. We show that our system can complete faces with large structural and appearance variations using a single feed-forward pass of computation with mean inference time of 0.007 seconds for images at 1024 x 1024 resolution. We also perform a pilot human study that shows our approach outperforms state-of-the-art face completion methods in terms of rank analysis. The code will be released upon publication.
△ Less
Submitted 23 January, 2018;
originally announced January 2018.
-
Probing the QCD Critical Point with Relativistic Heavy-Ion Collisions
Authors:
Steffen A. Bass,
Hannah Petersen,
Cory Quammen,
Hal Canary,
Christopher G. Healey,
Russell M. Taylor II
Abstract:
We utilize an event-by-event relativistic hydrodynamic calculation performed at a number of different incident beam energies to investigate the creation of hot and dense QCD matter near the critical point. Using state-of-the-art analysis and visualization tools we demonstrate that each collision event probes QCD matter characterized by a wide range of temperatures and baryo-chemical potentials, ma…
▽ More
We utilize an event-by-event relativistic hydrodynamic calculation performed at a number of different incident beam energies to investigate the creation of hot and dense QCD matter near the critical point. Using state-of-the-art analysis and visualization tools we demonstrate that each collision event probes QCD matter characterized by a wide range of temperatures and baryo-chemical potentials, making a dynamical response of the system to the vicinity of the critical point very difficult to isolate above the background.
△ Less
Submitted 31 January, 2012;
originally announced February 2012.
