Asking Clarifying Questions Based on Negative Feedback in Conversational Search

Users often need to look through multiple search result pages or reformulate queries when they have complex information-seeking needs. Conversational search systems make it possible to improve user satisfaction by asking questions to clarify users’ search intents. This, however, can take significant effort to answer a series of questions starting with “what/why/how”. To quickly identify user intent and reduce effort during interactions, we propose an intent clarification task based on yes/no questions where the system needs to ask the correct question about intents within the fewest conversation turns. In this task, it is essential to use negative feedback about the previous questions in the conversation history. To this end, we propose a Maximum-Marginal-Relevance (MMR) based BERT model (MMR-BERT) to leverage negative feedback based on the MMR principle for the next clarifying question selection. Experiments on the Qulac dataset show that MMR-BERT outperforms state-of-the-art baselines significantly on the intent identification task and the selected questions also achieve significantly better performance in the associated document retrieval tasks.

Introduction. In traditional Web search, users with complex information needs often need to look through multiple pages or reformulate queries to find their target information. In recent years, intelligent assistants such as Google Now, Apple Siri, or Microsoft Cortana make it possible for the system to interact with users through conversations. By asking questions to clarify ambiguous, faceted, or incomplete queries, conversational search systems could improve user satisfaction with better search quality. Thus, how to ask clarifying questions has become an important research topic. There are two typical types of clarifying questions: special questions beginning with what/why/how etc. and general (yes/no) questions that can be answered with “yes” or “no”. Special questions often let a user give specific information about a query such as “What do you want to know about COVID-19?” for the user query “COVID-19”. This kind of question is usually more difficult and requires more user effort to answer than questions such as “Do you want to know the symptoms of COVID-19?”

Discussion / Conclusion. In this paper, we propose an intent clarification task based on yes/no clarifying questions in information-seeking conversations. The task’s goal is to ask questions that can uncover the true user intent behind an ambiguous or faced query within the fewest conversation turns. We propose a maximal-marginal-relevance-based BERT model (MMR-BERT) that leverages the negative feedback to the previous questions using the MMR principle. Experimental results on the refined Qulac dataset show that MMR-BERT has significantly better performance than the competing question selection models in both the intent identification task and the associated document retrieval task. For future work, we plan to evaluate the effect of the asked clarifying questions on the associated document retrieval task with neural document retrieval models. We are also interested in studying how to effectively use negative feedback on the clarifying questions in the document retrieval model.