-
LLM Roleplay: Simulating Human-Chatbot Interaction
Authors:
Hovhannes Tamoyan,
Hendrik Schuff,
Iryna Gurevych
Abstract:
The development of chatbots requires collecting a large number of human-chatbot dialogues to reflect the breadth of users' sociodemographic backgrounds and conversational goals. However, the resource requirements to conduct the respective user studies can be prohibitively high and often only allow for a narrow analysis of specific dialogue goals and participant demographics. In this paper, we prop…
▽ More
The development of chatbots requires collecting a large number of human-chatbot dialogues to reflect the breadth of users' sociodemographic backgrounds and conversational goals. However, the resource requirements to conduct the respective user studies can be prohibitively high and often only allow for a narrow analysis of specific dialogue goals and participant demographics. In this paper, we propose LLM-Roleplay: a goal-oriented, persona-based method to automatically generate diverse multi-turn dialogues simulating human-chatbot interaction. LLM-Roleplay can be applied to generate dialogues with any type of chatbot and uses large language models (LLMs) to play the role of textually described personas. To validate our method we collect natural human-chatbot dialogues from different sociodemographic groups and conduct a human evaluation to compare real human-chatbot dialogues with our generated dialogues. We compare the abilities of state-of-the-art LLMs in embodying personas and holding a conversation and find that our method can simulate human-chatbot dialogues with a high indistinguishability rate.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning
Authors:
Lili Yu,
Bowen Shi,
Ramakanth Pasunuru,
Benjamin Muller,
Olga Golovneva,
Tianlu Wang,
Arun Babu,
Binh Tang,
Brian Karrer,
Shelly Sheynin,
Candace Ross,
Adam Polyak,
Russell Howes,
Vasu Sharma,
Puxin Xu,
Hovhannes Tamoyan,
Oron Ashual,
Uriel Singer,
Shang-Wen Li,
Susan Zhang,
Richard James,
Gargi Ghosh,
Yaniv Taigman,
Maryam Fazel-Zarandi,
Asli Celikyilmaz
, et al. (2 additional authors not shown)
Abstract:
We present CM3Leon (pronounced "Chameleon"), a retrieval-augmented, token-based, decoder-only multi-modal language model capable of generating and infilling both text and images. CM3Leon uses the CM3 multi-modal architecture but additionally shows the extreme benefits of scaling up and tuning on more diverse instruction-style data. It is the first multi-modal model trained with a recipe adapted fr…
▽ More
We present CM3Leon (pronounced "Chameleon"), a retrieval-augmented, token-based, decoder-only multi-modal language model capable of generating and infilling both text and images. CM3Leon uses the CM3 multi-modal architecture but additionally shows the extreme benefits of scaling up and tuning on more diverse instruction-style data. It is the first multi-modal model trained with a recipe adapted from text-only language models, including a large-scale retrieval-augmented pre-training stage and a second multi-task supervised fine-tuning (SFT) stage. It is also a general-purpose model that can do both text-to-image and image-to-text generation, allowing us to introduce self-contained contrastive decoding methods that produce high-quality outputs. Extensive experiments demonstrate that this recipe is highly effective for multi-modal models. CM3Leon achieves state-of-the-art performance in text-to-image generation with 5x less training compute than comparable methods (zero-shot MS-COCO FID of 4.88). After SFT, CM3Leon can also demonstrate unprecedented levels of controllability in tasks ranging from language-guided image editing to image-controlled generation and segmentation.
△ Less
Submitted 5 September, 2023;
originally announced September 2023.
-
BARTSmiles: Generative Masked Language Models for Molecular Representations
Authors:
Gayane Chilingaryan,
Hovhannes Tamoyan,
Ani Tevosyan,
Nelly Babayan,
Lusine Khondkaryan,
Karen Hambardzumyan,
Zaven Navoyan,
Hrant Khachatrian,
Armen Aghajanyan
Abstract:
We discover a robust self-supervised strategy tailored towards molecular representations for generative masked language models through a series of tailored, in-depth ablations. Using this pre-training strategy, we train BARTSmiles, a BART-like model with an order of magnitude more compute than previous self-supervised molecular representations. In-depth evaluations show that BARTSmiles consistentl…
▽ More
We discover a robust self-supervised strategy tailored towards molecular representations for generative masked language models through a series of tailored, in-depth ablations. Using this pre-training strategy, we train BARTSmiles, a BART-like model with an order of magnitude more compute than previous self-supervised molecular representations. In-depth evaluations show that BARTSmiles consistently outperforms other self-supervised representations across classification, regression, and generation tasks setting a new state-of-the-art on 11 tasks. We then quantitatively show that when applied to the molecular domain, the BART objective learns representations that implicitly encode our downstream tasks of interest. For example, by selecting seven neurons from a frozen BARTSmiles, we can obtain a model having performance within two percentage points of the full fine-tuned model on task Clintox. Lastly, we show that standard attribution interpretability methods, when applied to BARTSmiles, highlight certain substructures that chemists use to explain specific properties of molecules. The code and the pretrained model are publicly available.
△ Less
Submitted 29 November, 2022;
originally announced November 2022.
