Many disciplines rely on economic games to measure prosocial behavior. However, there is a concern that participants may misunderstand these games, complicating interpretation of results. This study combines online and laboratory data (total n = 1568) to assess subject comprehension of five standard economic games: the Dictator Game, Ultimatum Game, Trust Game, Public Goods Game, and Prisoner's Dilemma. The online and lab data collections are carried out separately and for the online data collection we collect data for two separate platforms (Prolific and CloudResearch's MTurk Toolkit). Within each data collection participants carry out all five games, and are randomized to comprehension questions with or without incentives for correct answers. Results indicate that misunderstanding is common: the proportion of participants who misunderstood ranged from 22 % (Dictator Game) to 70 % (Trust Game) in the online samples and from 22 % (Dictator Game) to 53 % (Public Goods Game) in the lab sample. Incentivizing the comprehension questions had no significant impact on misunderstanding, but numeracy was associated with lower misunderstanding. Misunderstanding also predicted increased prosocial behavior in several of the games. Our findings suggest that misunderstanding may be important in explaining prosocial behavior, making it more complicated to draw clear inferences about social preferences from experimental data.