Ethics toolkits, checklists and workshops are intended to help integrate ethical considerations into the design of data-driven systems. Yet little is known about what long-term effect such integrations might have. We conducted an ethnographic investigation of the adoption of an internal ethical toolkit in a major European city organization. We find that neither toolkit designers nor organizations that implement these, pay attention to the affective experience and emotional costs of integrating ethics toolkits into technology design team workflows. We demonstrate how moral awareness, while necessary for moral technical practice, also leads to unaccounted moral stress for practitioners.