OpenAssistant Conversations -- Democratizing Large Language Model Alignment
Authors:
Andreas Köpf,
Yannic Kilcher,
Dimitri von Rütte,
Sotiris Anagnostidis,
Zhi-Rui Tam,
Keith Stevens,
Abdullah Barhoum,
Nguyen Minh Duc,
Oliver Stanley,
Richárd Nagyfi,
Shahul ES,
Sameer Suri,
David Glushkov,
Arnav Dantuluri,
Andrew Maguire,
Christoph Schuhmann,
Huu Nguyen,
Alexander Mattick
Abstract:
Aligning large language models (LLMs) with human preferences has proven to drastically improve usability and has driven rapid adoption as demonstrated by ChatGPT. Alignment techniques such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) greatly reduce the required skill and domain knowledge to effectively harness the capabilities of LLMs, increasing their acce…
▽ More
Aligning large language models (LLMs) with human preferences has proven to drastically improve usability and has driven rapid adoption as demonstrated by ChatGPT. Alignment techniques such as supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF) greatly reduce the required skill and domain knowledge to effectively harness the capabilities of LLMs, increasing their accessibility and utility across various domains. However, state-of-the-art alignment techniques like RLHF rely on high-quality human feedback data, which is expensive to create and often remains proprietary. In an effort to democratize research on large-scale alignment, we release OpenAssistant Conversations, a human-generated, human-annotated assistant-style conversation corpus consisting of 161,443 messages in 35 different languages, annotated with 461,292 quality ratings, resulting in over 10,000 complete and fully annotated conversation trees. The corpus is a product of a worldwide crowd-sourcing effort involving over 13,500 volunteers. Models trained on OpenAssistant Conversations show consistent improvements on standard benchmarks over respective base models. We release our code and data under a fully permissive licence.
△ Less
Submitted 31 October, 2023; v1 submitted 14 April, 2023;
originally announced April 2023.
Dequantization of electric charge: Probing scenarios of cosmological multi-component dark matter
Authors:
Duong Van Loi,
Nguyen Manh Duc,
Phung Van Dong
Abstract:
Since the electric charge in the standard model is theoretically not quantized, we may have a variant of it, called dark charge. Similar to the electric charge, the dark charge neither commutes nor closes algebraically with $SU(2)_L$. The condition of algebraic closure leads to a novel gauge extension, $SU(2)_L \otimes U(1)_Y \otimes U(1)_N$, where $Y$ and $N$ determine the electric and dark charg…
▽ More
Since the electric charge in the standard model is theoretically not quantized, we may have a variant of it, called dark charge. Similar to the electric charge, the dark charge neither commutes nor closes algebraically with $SU(2)_L$. The condition of algebraic closure leads to a novel gauge extension, $SU(2)_L \otimes U(1)_Y \otimes U(1)_N$, where $Y$ and $N$ determine the electric and dark charges, respectively, apart from the color group. We argue that the existence of the dark charge, thus $N$, leads to novel scenarios of multi-component dark matter, in general. The dark matter stability is determined by a residual (or dark charge) gauge symmetry isomorphic to an even $Z_k$ discrete group, where $k$ is specified dependent on the value of the neutrino dark charge. This residual symmetry divides the standard model particles into distinct classes, which possibly accommodate dark matter, but each dark matter candidate cannot decay due to the color and electric charge conservation. We analyze in detail three specific models according to $k=2,4,6$ and determine the simplest dark matter candidates. For small $U(1)_N$ coupling, the two-component dark matter scenarios implied by the dark charge successfully explain the dark matter relic density and the recent XENON1T excess, as well as the beam dump, neutrino scattering, and astrophysical bounds. Otherwise, for large $U(1)_N$ coupling, we have multi-WIMPs coexisted beyond the weak scale.
△ Less
Submitted 17 August, 2022; v1 submitted 23 June, 2021;
originally announced June 2021.
