We consider structural optimization (SO) under uncertainty formulated as a mathematical game between two players -- a "designer" and "nature". The first player wants to design a structure that performs optimally, whereas the second player tries to find the worst possible conditions to impose on the structure. Several solution concepts exist for such games, including Stackelberg and Nash equilibria and Pareto optima. Pareto optimality is shown not to be a useful solution concept. Stackelberg and Nash games are, however, both of potential interest, but these concepts are hardly ever discussed in the literature on SO under uncertainty. Based on concrete examples of topology optimization of trusses and finite element-discretized continua under worst-case load uncertainty, we therefore analyze and compare the two solution concepts. In all examples, Stackelberg equilibria exist and can be found numerically, but for some cases we demonstrate nonexistence of Nash equilibria. This motivates a view of the Stackelberg solution concept as the correct one. However, we also demonstrate that existing Nash equilibria can be found using a simple so-called decomposition algorithm, which could be of interest for other instances of SO under uncertainty, where it is difficult to find a numerically efficient Stackelberg formulation.