PhD Proposal: Teaching Machines to Ask Clarification Questions

Sudha Rao
05.12.2017 13:30 to 15:00
AVW 3450

An overarching goal of the natural language processing community is to develop techniques that would enable machines to process naturally occurring text as efficiently as humans do. However, as humans, we do not always understand each other. According to Gricean pragmatics, speakers and listeners adhere to a Cooperative Principle where a speaker communicates information that is as informative as required and not more. In case of gaps or mismatches in knowledge, the listener resorts to asking questions. With the advancements of artificial intelligence technologies, we increasingly find ourselves interacting with automated agents (like Apple's Siri, Amazon's Alexa, Google Home, etc) in naturally spoken languages. If we wish to make such human-bot interactions as efficient as human-human interactions are, it is important that we teach machines to ask clarification questions when faced with uncertainty or knowledge gaps.

In the field of natural language processing, however, despite decades of work on question-answering, there has been little work in question asking. Moreover, most of the previous work has focused on generating reading comprehension style questions which, by definition, are answerable from the provided text. The goal of my dissertation work, on the other hand, is to teach machines to ask clarification questions pointing out the missing information in a text. Primarily, I focus on two scenarios where I find such question asking to be useful: (1) clarification questions on posts found in community-driven Q&A forums like StackExchange (2) clarification questions during goal-oriented dialogue sessions.

In my first line of research, I study the problem of question generation using data from StackExchange, a plentiful online resource in which people routinely ask clarifying questions to posts so that they can better offer assistance to the original poster. In my preliminary work, I created a novel dataset using StackExchange and addressed the question selection problem, more specifically select the right clarification question from a set of prior questions. I developed a novel neural network model inspired by the notion of expected value of perfect information: a good question is the one whose expected answer is going to be the most useful. In my first proposed work, I plan to use the following two strategies to build more generalizable systems: (a) a template based question generation model and (b) an encoder-decoder based neural generative model.

In my second line of research, I study the problem of question generation using the Ubuntu Dialogue Corpus, a two-way conversational data extracted systematically from chat logs where people discuss issues with their Ubuntu Operating system. In my preliminary work, I addressed the task of predicting the next best response given a context of a conversation. I built a novel neural network model inspired by the best practices in dialogue modeling coupled with our novel vocabulary selection strategies. In my second proposed work, I plan to explore how can I generate a clarification question as the next response, given different levels of context of a conversation.
In both the research agendas described so far, I took a purely corpus-driven approach to generating clarification questions i.e. learning to ask a question by looking at previously asked questions in a similar context. However, inferring a knowledge gap requires a certain level of domain knowledge that is currently lacking in our proposed models. Therefore, in my third proposed work, I plan to explore how can we use external knowledge sources to understand what is missing in a given context and then ask a clarification question.

Examining Committee:

Chair: Dr. Hal Daumé III

Dept rep: Dr. David Jacobs

Members: Dr. Philip Resnik

Dr. Lucy Vanderwende