Problems of cooperation have often been simplified as the choice between defection and cooperation, although in many empirical situations it is also possible to walk away from the interaction. We present the results of two lab-in-the-field experiments with a diverse pool of subjects who play optional and compulsory public goods games both with and without punishment. We find that the most important institution to foster cooperation is punishment, which is more effective in a compulsory game. In contrast to Rand and Nowak (Nat Commun 2(1):1-7, 2011), we find that loners are not responsible for anti-social punishment, which is mostly imputable to low-contributors (free-riders). Loners neither totally free-ride nor they significantly punish cooperators (or other types of players): they simply avoid all forms of participation whenever possible.