Peer review is the mechanism for quality control in academic journals. When a manuscript is submitted to a journal, the editors invite other researchers – peers – to review it anonymously. The reviews should serve to support the journal editors in making decisions, and to support the authors in improving the manuscripts before publication. Therefore, reviews need to be fair and constructive. As reviewing can also take considerable effort, it is useful for the reviewer to consider how to do it effectively. Given the important role of peer review in a field, and the considerable effort it takes, it is valuable to jointly consider all these aspects of reviewing in a dialogue with reviewers, authors and editors. This paper presents the outcomes of such a dialogue with 49 participants in the field of engineering education research.