Showing 1–1 of 1 results for author: Anand, B B P

Looking Under the Hood : Tools for Diagnosing your Question Answering Engine

Authors: Eric Breck, Marc Light, Gideon S. Mann, Ellen Riloff, Brianne Brown Pranav Anand, Mats Rooth, Michael Thelen

Abstract: In this paper we analyze two question answering tasks : the TREC-8 question answering task and a set of reading comprehension exams. First, we show that Q/A systems perform better when there are multiple answer opportunities per question. Next, we analyze common approaches to two subproblems: term overlap for answer sentence identification, and answer ty** for short answer extraction. We prese… ▽ More In this paper we analyze two question answering tasks : the TREC-8 question answering task and a set of reading comprehension exams. First, we show that Q/A systems perform better when there are multiple answer opportunities per question. Next, we analyze common approaches to two subproblems: term overlap for answer sentence identification, and answer ty** for short answer extraction. We present general tools for analyzing the strengths and limitations of techniques for these subproblems. Our results quantify the limitations of both term overlap and answer ty** to distinguish between competing answer candidates. △ Less

Submitted 3 July, 2001; originally announced July 2001.

Comments: Revision of paper appearing in the Proceedings of the Workshop on Open-Domain Question Answering

ACM Class: I.2.7