Neural Generation Meets Real People: Building a Social, Informative Open-Domain Dialogue Agent
Authors:
Ethan A. Chi,
Ashwin Paranjape,
Abigail See,
Caleb Chiam,
Trenton Chang,
Kathleen Kenealy,
Swee Kiat Lim,
Amelia Hardy,
Chetanya Rastogi,
Haojun Li,
Alexander Iyabor,
Yutong He,
Hari Sowrirajan,
Peng Qi,
Kaushik Ram Sadagopan,
Nguyet Minh Phu,
Dilara Soylu,
Jillian Tang,
Avanika Narayan,
Giovanni Campagna,
Christopher D. Manning
Abstract:
We present Chirpy Cardinal, an open-domain social chatbot. Aiming to be both informative and conversational, our bot chats with users in an authentic, emotionally intelligent way. By integrating controlled neural generation with scaffolded, hand-written dialogue, we let both the user and bot take turns driving the conversation, producing an engaging and socially fluent experience. Deployed in the…
▽ More
We present Chirpy Cardinal, an open-domain social chatbot. Aiming to be both informative and conversational, our bot chats with users in an authentic, emotionally intelligent way. By integrating controlled neural generation with scaffolded, hand-written dialogue, we let both the user and bot take turns driving the conversation, producing an engaging and socially fluent experience. Deployed in the fourth iteration of the Alexa Prize Socialbot Grand Challenge, Chirpy Cardinal handled thousands of conversations per day, placing second out of nine bots with an average user rating of 3.58/5.
△ Less
Submitted 16 January, 2023; v1 submitted 25 July, 2022;
originally announced July 2022.
Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality
Authors:
Mina Lee,
Chris Donahue,
Robin Jia,
Alexander Iyabor,
Percy Liang
Abstract:
We release a new benchmark for lexical substitution, the task of finding appropriate substitutes for a target word in a context. To assist humans with writing, lexical substitution systems can suggest words that humans cannot easily think of. However, existing benchmarks depend on human recall as the only source of data, and therefore lack coverage of the substitutes that would be most helpful to…
▽ More
We release a new benchmark for lexical substitution, the task of finding appropriate substitutes for a target word in a context. To assist humans with writing, lexical substitution systems can suggest words that humans cannot easily think of. However, existing benchmarks depend on human recall as the only source of data, and therefore lack coverage of the substitutes that would be most helpful to humans. Furthermore, annotators often provide substitutes of low quality, which are not actually appropriate in the given context. We collect higher-coverage and higher-quality data by framing lexical substitution as a classification problem, guided by the intuition that it is easier for humans to judge the appropriateness of candidate substitutes than conjure them from memory. To this end, we use a context-free thesaurus to produce candidates and rely on human judgement to determine contextual appropriateness. Compared to the previous largest benchmark, our Swords benchmark has 4.1x more substitutes per target word for the same level of quality, and its substitutes are 1.5x more appropriate (based on human judgement) for the same number of substitutes.
△ Less
Submitted 12 June, 2021; v1 submitted 8 June, 2021;
originally announced June 2021.
