This paper explores how humans interpret displays of emotion pro- duced by a social robot in real world situated interaction. Taking a multimodal conversation analytic approach, we analyze video data of families interacting with a Cozmo robot in their homes. Focusing on one happy and one sad robot animation, we study, on a turn-by-turn basis, how participants respond to audible and visible robot behavior designed to display emotion. We show how emotion animations are consequential for interactional progres- sivity: While displays of happiness typically move the interaction forward, displays of sadness regularly lead to a reconsideration of previous actions by humans. Furthermore, in making sense of the robot animations people may move beyond the designer’s re- ported intentions, actually broadening the opportunities for their subsequent engagement. We discuss how sadness functions as an interactional "rewind button" and how the inherent vagueness of emotion displays can be deployed in design.

Delays in the completion of joint actions are sometimes unavoidable. How should a robot communicate that it cannot immediately act or respond in a collaborative task? Drawing on video recordings of a face scanning activity in family homes, we investigate how humans make sense of a Cozmo robot’s delays on a moment-by-moment basis. Cozmo’s sounds and embodied actions are recognized as indicators of delay but encourage human participants to act in ways that undermine the scanning process. In comparing the robot’s delay management strategies with human-human vocal and embodied practices, we demonstrate key differences in the sequences that impact how the robot is understood. The study demonstrates how delay events are accomplished as embodied displays that are distributed across co-participants. We present a framework for making delay transparent through situated explanations, particularly in the form of non-lexical sounds and bodily actions.

Smooth traffic presupposes fine coordination between different actors, such as pedestrians, cyclists and car drivers. When autonomous vehicles join regular traffic, they need to coordinate with humans on the road. Prior work has often studied and designed for interaction with autonomous vehicles in structured environments such as traffic intersections. This paper describes aspects of coordination also in less structured situations during mundane maneuvers such as overtaking. Taking an ethnomethodological and conversation analytic approach, the paper analyzes video recordings of self-driving shuttle buses in Sweden. Initial findings suggest that the shuttle buses currently do not comply with cyclists’ expectations of social coordination in traffic. The paper highlights that communication and coordination with human road users is crucial for smooth flow of traffic and successful deployment of autonomous vehicles also in less structured traffic environments.

Horns and sirens are important tools for communicating on the road, which are still understudied in autonomous vehicles. While HRI has explored different ways in which robots could sound, we focus on the range of actions that a single sound can accomplish in interaction. In a Research through Design study involving autonomous shuttle buses in public transport, we explored sound design with the help of voice-overs to video recordings of the buses on the road and Wizard-of-Oz tests in live traffic. The buses are slowed down by (unnecessary) braking in response to people getting close. We found that prolonged jingles draw attention to the bus and invite interaction, while repeated short beeps and bell sounds can instruct the movement of others away from the bus. We highlight the importance of designing sound in sequential interaction and describe a new method for embedding video interaction analysis in the design process.