The field of Intelligent Virtual Agents (IVAs) has evolved immensely with respect to design and development of agents over the years. This has brought the following questions: What processes and methods are used to design virtual agents, and in particular, to what extent and how are users involved in the design process of virtual agents? In this paper, we review papers from the conference “Intelligent Virtual Agents” for the last five years (2015 - 2019) to shed light on these questions. The review included 308 short and long papers, with a focus on the interactive aspects of design. The review showed that only 14% of 308 papers explicitly mentioned/referred to user participation during the design of an IVA. User involvement is classified into two categories: one-time and iterative. The few studies that mention design only report on the use of standard Human-Computer Interaction (HCI) design methods to a rather limited degree.