“Hello There! Is Now a Good Time to Talk?”: Opportune Moments for Proactive Interactions with Smart Speakers

Increasing number of researchers and designers are envisioning a wide range of novel proactive conversational services for smart speakers such as context-aware reminders and restocking household items. When initiating conversational interactions proactively, smart speakers need to consider users’ contexts to minimize disruption. In this work, we aim to broaden our understanding of opportune moments for proactive conversational interactions in domestic contexts. Toward this goal, we built a voice-based experience sampling device and conducted a one-week field study with 40 participants living in university dormitories. From 3,572 in-situ user experience reports, we proposed 19 activity categories to investigate contextual factors related to interruptibility. Our data analysis results show that the key determinants for opportune moments are closely related to both personal contextual factors such as busyness, mood, and resource conflicts for dual-tasking, and the other contextual factors associated with the everyday routines at home, including user mobility and social presence. Based on these findings, we discuss the need for designing context-aware proactive conversation management features that dynamically control conversational interactions based on users’ contexts and routines.

Introduction. Smart speakers come with an intelligent voice assistant feature that supports speech-based interactions for a wide range of tasks, including time checking, delivery tracking, and QnA. Currently, smart speaker services are mostly reactive to user’s commands. Recently, Amazon and Google with top shares in the smart speaker market started to consider proactive services such as reminding a user’s schedule [59] and supporting home safety and security [52]. Prior studies have already demonstrated the usefulness of various proactive services: context-aware reminders/recommendations [10, 57, 63], and self-tracking/reflection for productivity [30, 68] and mental well-being [14, 35]. While proactive services provide useful information for inspiring and engaging users, prior studies also warned that timing and relevance of proactive services are critical to the user experience [5, 11].

Discussion / Conclusion. We review our findings in relation to prior studies, present design insights for designing context-aware proactive conversation management system for smart speakers, and discuss limitations of our work. Our study empirically examined three contextual factors relevant to the interruptibility of smart speaker-based proactive services in domestic contexts: personal, movement-related, and social factors. These factors show different characteristics compared to the factors affecting the interruptibility of services provided in conventional computing contexts, such as computers and mobile devices in offices and vehicles. The personal contextual factors include concentration and engagement, urgency and busyness, psychological/physical states (e.g., wakefulness, mood, and fatigue), and auditory/verbal channel availability. Our results are consistent with prior studies on interruptibility; users engaged in challenging tasks are less interruptible [46] than those participating in rote tasks [37].