BACKGROUND AND PURPOSE: Brain gadolinium retention is consistently reported for linear gadolinium-based contrast agents, while the results for macrocyclics are contradictory and potential clinical manifestations remain controversial. Furthermore, most previous studies are based on conventional T1-weighted MR imaging. We therefore aimed to quantitatively investigate longitudinal and transversal relaxation in the brain in relation to previous gadolinium-based contrast agent administration and explore associations with disability in multiple sclerosis. MATERIALS AND METHODS: Eighty-five patients with MS and 21 healthy controls underwent longitudinal and transverse relaxation rate (R-1 and R-2) relaxometry. Patients were divided into linear, mixed, and macrocyclic groups based on previous gadolinium-based contrast agent administration. Neuropsychological testing was performed in 53 patients. The dentate nucleus, globus pallidus, caudate nucleus, and thalamus were manually segmented. Repeatability measures were also performed. RESULTS: The relaxometry was robust (2.0% scan-rescan difference) and detected higher R-1 (dentate nucleus, globus pallidus, caudate nucleus, thalamus) and R-2 (globus pallidus, caudate nucleus) in patients receiving linear gadolinium-based contrast agents compared with controls. The number of linear gadolinium-based contrast agent administrations was associated with higher R-1 and R-2 in all regions (except R-2 in the thalamus). No similar differences and associations were found for the macrocyclic group. Higher relaxation was associated with lower information-processing speed (dentate nucleus, thalamus) and verbal fluency (caudate nucleus, thalamus). No associations were found with physical disability or fatigue. CONCLUSIONS: Previous linear, but not macrocyclic, gadolinium-based contrast agent administration is associated with higher relaxation rates in a dose-dependent manner. Higher relaxation in some regions is associated with cognitive impairment but not physical disability or fatigue in MS. The findings should be interpreted with care but encourage studies into gadolinium retention and cognition.