-
Biomedical knowledge graph-optimized prompt generation for large language models
Authors:
Karthik Soman,
Peter W Rose,
John H Morris,
Rabia E Akbas,
Brett Smith,
Braian Peetoom,
Catalina Villouta-Reyes,
Gabriel Cerono,
Yongmei Shi,
Angela Rizk-Jackson,
Sharat Israni,
Charlotte A Nelson,
Sui Huang,
Sergio E Baranzini
Abstract:
Large Language Models (LLMs) are being adopted at an unprecedented rate, yet still face challenges in knowledge-intensive domains like biomedicine. Solutions such as pre-training and domain-specific fine-tuning add substantial computational overhead, requiring further domain expertise. Here, we introduce a token-optimized and robust Knowledge Graph-based Retrieval Augmented Generation (KG-RAG) fra…
▽ More
Large Language Models (LLMs) are being adopted at an unprecedented rate, yet still face challenges in knowledge-intensive domains like biomedicine. Solutions such as pre-training and domain-specific fine-tuning add substantial computational overhead, requiring further domain expertise. Here, we introduce a token-optimized and robust Knowledge Graph-based Retrieval Augmented Generation (KG-RAG) framework by leveraging a massive biomedical KG (SPOKE) with LLMs such as Llama-2-13b, GPT-3.5-Turbo and GPT-4, to generate meaningful biomedical text rooted in established knowledge. Compared to the existing RAG technique for Knowledge Graphs, the proposed method utilizes minimal graph schema for context extraction and uses embedding methods for context pruning. This optimization in context extraction results in more than 50% reduction in token consumption without compromising the accuracy, making a cost-effective and robust RAG implementation on proprietary LLMs. KG-RAG consistently enhanced the performance of LLMs across diverse biomedical prompts by generating responses rooted in established knowledge, accompanied by accurate provenance and statistical evidence (if available) to substantiate the claims. Further benchmarking on human curated datasets, such as biomedical true/false and multiple-choice questions (MCQ), showed a remarkable 71% boost in the performance of the Llama-2 model on the challenging MCQ dataset, demonstrating the framework's capacity to empower open-source models with fewer parameters for domain specific questions. Furthermore, KG-RAG enhanced the performance of proprietary GPT models, such as GPT-3.5 and GPT-4. In summary, the proposed framework combines explicit and implicit knowledge of KG and LLM in a token optimized fashion, thus enhancing the adaptability of general-purpose LLMs to tackle domain-specific questions in a cost-effective fashion.
△ Less
Submitted 13 May, 2024; v1 submitted 28 November, 2023;
originally announced November 2023.
-
Ten Simple Rules for Reproducible Research in Jupyter Notebooks
Authors:
Adam Rule,
Amanda Birmingham,
Cristal Zuniga,
Ilkay Altintas,
Shih-Cheng Huang,
Rob Knight,
Niema Moshiri,
Mai H. Nguyen,
Sara Brin Rosenthal,
Fernando Pérez,
Peter W. Rose
Abstract:
Reproducibility of computational studies is a hallmark of scientific methodology. It enables researchers to build with confidence on the methods and findings of others, reuse and extend computational pipelines, and thereby drive scientific progress. Since many experimental studies rely on computational analyses, biologists need guidance on how to set up and document reproducible data analyses or s…
▽ More
Reproducibility of computational studies is a hallmark of scientific methodology. It enables researchers to build with confidence on the methods and findings of others, reuse and extend computational pipelines, and thereby drive scientific progress. Since many experimental studies rely on computational analyses, biologists need guidance on how to set up and document reproducible data analyses or simulations.
In this paper, we address several questions about reproducibility. For example, what are the technical and non-technical barriers to reproducible computational studies? What opportunities and challenges do computational notebooks offer to overcome some of these barriers? What tools are available and how can they be used effectively?
We have developed a set of rules to serve as a guide to scientists with a specific focus on computational notebook systems, such as Jupyter Notebooks, which have become a tool of choice for many applications. Notebooks combine detailed workflows with narrative text and visualization of results. Combined with software repositories and open source licensing, notebooks are powerful tools for transparent, collaborative, reproducible, and reusable data analyses.
△ Less
Submitted 13 October, 2018;
originally announced October 2018.
-
The Q_weak Experimental Apparatus
Authors:
Qweak Collaboration,
T. Allison,
M. Anderson,
D. Androic,
D. S. Armstrong,
A. Asaturyan,
T. D. Averett,
R. Averill,
J. Balewski,
J. Beaufait,
R. S. Beminiwattha,
J. Benesch,
F. Benmokhtar,
J. Bessuille,
J. Birchall,
E. Bonnell,
J. Bowman,
P. Brindza,
D. B. Brown,
R. D. Carlini,
G. D. Cates,
B. Cavness,
G. Clark,
J. C. Cornejo,
S. Covrig Dusa
, et al. (104 additional authors not shown)
Abstract:
The Jefferson Lab Q_weak experiment determined the weak charge of the proton by measuring the parity-violating elastic scattering asymmetry of longitudinally polarized electrons from an unpolarized liquid hydrogen target at small momentum transfer. A custom apparatus was designed for this experiment to meet the technical challenges presented by the smallest and most precise ${\vec{e}}$p asymmetry…
▽ More
The Jefferson Lab Q_weak experiment determined the weak charge of the proton by measuring the parity-violating elastic scattering asymmetry of longitudinally polarized electrons from an unpolarized liquid hydrogen target at small momentum transfer. A custom apparatus was designed for this experiment to meet the technical challenges presented by the smallest and most precise ${\vec{e}}$p asymmetry ever measured. Technical milestones were achieved at Jefferson Lab in target power, beam current, beam helicity reversal rate, polarimetry, detected rates, and control of helicity-correlated beam properties. The experiment employed 180 microA of 89% longitudinally polarized electrons whose helicity was reversed 960 times per second. The electrons were accelerated to 1.16 GeV and directed to a beamline with extensive instrumentation to measure helicity-correlated beam properties that can induce false asymmetries. Moller and Compton polarimetry were used to measure the electron beam polarization to better than 1%. The electron beam was incident on a 34.4 cm liquid hydrogen target. After passing through a triple collimator system, scattered electrons between 5.8 degrees and 11.6 degrees were bent in the toroidal magnetic field of a resistive copper-coil magnet. The electrons inside this acceptance were focused onto eight fused silica Cerenkov detectors arrayed symmetrically around the beam axis. A total scattered electron rate of about 7 GHz was incident on the detector array. The detectors were read out in integrating mode by custom-built low-noise pre-amplifiers and 18-bit sampling ADC modules. The momentum transfer Q^2 = 0.025 GeV^2 was determined using dedicated low-current (~100 pA) measurements with a set of drift chambers before (and a set of drift chambers and trigger scintillation counters after) the toroidal magnet.
△ Less
Submitted 6 January, 2015; v1 submitted 24 September, 2014;
originally announced September 2014.
