Objective: This study aims to investigate whether antidepressant users display differences in fat distribution and muscle composition relative to non-users and to explore risk factors for developing cardiovascular disease (CVD) and type 2 diabetes. Methods: The study used quantitative adipose and muscle tissue measures derived from magnetic resonance imaging data from UK Biobank (N = 40,174). Fat distribution and muscle composition of selective serotonin reuptake inhibitor (SSRI) and tricyclic antidepressant (TCA) users were compared with sex-, age-, and BMI-matched control individuals. Cox regression models were used to test for increased risk of developing CVD and type 2 diabetes. Results: SSRI users had more visceral fat, smaller muscle volume, and higher muscle fat infiltration compared with matched control individuals. Female users showed a larger increase in BMI over time compared with male users. However, male users displayed an unhealthier body composition profile. Male SSRI users also had an increased risk of developing CVD. Both male and female TCA users showed lower muscle volume and an increased risk of developing type 2 diabetes. Conclusions: Adverse changes in body composition of antidepressant users are not captured by tracking the body weight or the BMI of the patients. These changes may lead to a worsened cardiometabolic risk profile.