Objective: Ovarian removal prior to spontaneous/natural menopause (SM) is associated with increased risk of late life dementias including Alzheimer's disease. This increased risk may be related to the sudden and early loss of endogenous estradiol. Women with breast cancer gene mutations (BRCAm) are counseled to undergo oophorectomy prior to SM to significantly reduce their risk of developing breast, ovarian, and cervical cancers. There is limited evidence of the neurological effects of ovarian removal prior to the age of SM showing women without the BRCAm had cortical thinning in medial temporal lobe structures. A second study in women with BRCAm and bilateral salpingo-oophorectomy (BSO) noted changes in cognition. Methods: The present, cross-sectional study examined whole-brain differences in gray matter (GM) volume using high-resolution, quantitative magnetic resonance imaging in women with BRCAm and intact ovaries (BRCA-preBSO [study cohort with BRCA mutation prior to oophorectomy]; n = 9) and after surgery with (BSO + estradiol-based therapy [ERT]; n = 10) and without (BSO; n = 10) postsurgical estradiol hormone therapy compared with age-matched women (age-matched controls; n = 10) with their ovaries. Results: The BRCA-preBSO and BSO groups showed significantly lower GM volume in the left medial temporal and frontal lobe structures. BSO + ERT exhibited few areas of lower GM volume compared with age-matched controls. Novel to this study, we also observed that all three BRCAm groups exhibited significantly higher GM volume compared with age-matched controls, suggesting continued plasticity. Conclusions: The present study provides evidence, through lower GM volume, to support both the possibility that the BRCAm, alone, and early life BSO may play a role in increasing the risk for late-life dementia. At least for BRCAm with BSO, postsurgical ERT seems to ameliorate GM losses.