This work focuses on the reliability and resilience aspects of federated learning (FL) over wireless networks with Over-the-Air (OtA) aggregation of local model updates from distributed users. We propose a hierarchical FL architecture where OtA computation is used for intra-cluster aggregation and digital transmission is used for global aggregation. In addition, robust aggregation techniques are applied during global aggregation to eliminate the effect of potential perturbations from byzantine attackers. We show that this hierarchical design can combine the advantages of resource efficiency and system resilience. Furthermore, we observe some interesting interplay between communication cost, aggregation accuracy and byzantine resilience when allowing spatial reuse of frequency-time resources among spatially separated clusters.