Question Types in Digital Reference: An Evaluation of Question Taxonomies

Jeffrey Pomerantz School of Information Studies, Syracuse University, Syracuse, NY, 13244. Email: jppomera@syr.edu.

This study evaluates four taxonomies of question types to determine the expressiveness of each for questions received by digital reference services. The result is a faceted classification scheme that can be used as a basis for automating parts of the digital reference question answering process.

Introduction The fields of digital reference and question answering (Q&A) are moving from different directions towards similar goals: digital reference towards automated answering of reference questions, and Q&A towards automated answer formulation in increasingly open domains. As Q&A systems move towards increasingly open domains, the types of questions that they will be required to handle are likely to become more complex. Reference is the ultimate open-domain question answering environment, and therefore the ideal environment for developing such Q&A systems; the types of questions that may be received by these services is constrained only by the imagination of the patrons. The goal of this study is to bridge this gap: to suggest ways to bring ideas, tools, and methods from Q&A research and development to the problem of automating the reference process, and to bring the experience of the digital reference community to bear on the problem of open- domain Q&A. In order to accomplish this, it is necessary to be able to identify the type of a question. This study asks the following research questions: 1. What types of questions are received by digital

2. How can these question types be organized for reference services?

question answering system development?

Methodology The methodology for this study follows the following steps: (1) Identifying existing question taxonomies from the literature on library reference and question answering, (2) Classifying questions received by digital reference services according to these taxonomies, (3) Evaluating these taxonomies to determine the expressiveness of each for digital reference questions, and (4) Modifying these taxonomies to improve their expressiveness for digital reference questions.

Resu Its This study has identified four types of question taxonomies in the literature on reference and Q&A. These taxonomies classify questions according to: (1) the wh- word in the question, (2) the subject of the question, ( 3 ) the function of the expected answer, and (4) the form of the expected answer. These taxonomies correspond to the syntactic, semantic, discourse, and pragmatic levels of linguistic analysis, respectively. No one of these taxonomies describes a question sufficiently for the purposes of open- domain Q&A. However, as a faceted scheme - a combination of classification schemes, each of which views questions from a different perspective - these taxonomies provide a far richer and more powerful method for describing questions.

Discussion The recent movement in the Q&A community to develop systems to answer questions in increasingly open domains calls for systems that encompass world knowledge. Reference work has always utilized world knowledge - both of the library’s collection and of the practice of providing reference. A faceted classification scheme will enable Q&A systems to collect more information about the question than is currently possible, by enabling the conversion of questions into logical representations along multiple levels of linguistic analysis. A faceted scheme will also enable inference rules to be established about question types based on multiple dimensions of the question - inferences about the complexity of the question, potential answer sources, etc. Bringing world knowledge from reference work to Q&A system development will enable systems to be built to answer questions in broader domains. Bringing Q&A systems to digital reference wiIl enable these services to move towards automated answering of reference questions.

ASIST 2002 Poster 542