"It doesn't look good for a date": Transforming Critiques into Preferences for Conversational Recommendation Systems

Conversations aimed at determining good recommendations are iterative in nature. People often express their preferences in terms of a critique of the current recommendation (e.g., “It doesn’t look good for a date”), requiring some degree of common sense for a preference to be inferred. In this work, we present a method for transforming a user critique into a positive preference (e.g., “I prefer more romantic”) in order to retrieve reviews pertaining to potentially better recommendations (e.g., “Perfect for a romantic dinner”). We leverage a large neural language model (LM) in a fewshot setting to perform critique-to-preference transformation, and we test two methods for retrieving recommendations: one that matches embeddings, and another that fine-tunes an LM for the task. We instantiate this approach in the restaurant domain and evaluate it using a new dataset of restaurant critiques. In an ablation study, we show that utilizing critiqueto-preference transformation improves recommendations, and that there are at least three general cases that explain this improved performance.

Introduction. Conversational recommendation systems (CRSs) are dialog-based systems that aim to refine a set of options over multiple turns of a conversation, envisioning more natural interactions and better user modeling than in non-conversational approaches. However, the resulting dialogs still do not necessarily reflect how real conversations unfold. Most CRSs fall into two categories: they either frame the problem as a slot-filling task within a predefined feature space, such as Sun and Zhang (2018); Zhang et al. (2018); Budzianowski et al. (2018), which is closer to how people make decisions but not as flexible as real conversations; or they elicit preferences by asking users to rate specific items, such as Christakopoulou et al. (2016), which is in- When we examine situations involving real human agents (Lyu et al., 2021), decisions typically require multiple rounds of recommendations by the agent and critiques by the user, with the agent continuously improving the recommendations based upon user preferences that can be inferred from such critiques.

Discussion / Conclusion. In this paper, we presented an open-ended approach to content-based recommendations for CRS. We developed a novel critique interpretation method that uses GPT3 to infer positive preferences from freeform critiques. We also developed two methods for retrieving recommendations: one that matches embeddings and another that fine-tunes BERT for the task. We ran two ablation studies to test if transforming critiques into positive preferences would yield better recommendations, confirming that it improves performance across both methods. Finally, we described three critique patterns that cause systematic errors in recommendation search if critique interpretation is turned off. For future work, we will strive to use critiques to identify and remove unsuitable restaurants; we speculate that the sparsity of customer reviews generally makes it harder to “rule out” than to “rule in.” We will also study other issues such as when to ask clarification questions to resolve ambiguity in the scope of a critique.