Epicardial adipose tissue (EAT) has been recognized as a risk factor and independent predictor for cardiovascular diseases (CVDs), due to its intimate relationship with the myocardium and coronary arteries. Dixon MRI is widely used to depict adipose tissue by deriving fat and water signals. The purpose of this study was to automatically segment and quantify EAT from Dixon MRI data using a fully automated deep learning pipeline based on fat maps (FM-Net). Data used in this study was from a sub-study (HEALTH) of the Swedish CArdioPulmonarybiolmage Study (SCAPIS), with 6504 Dixon MRI 2D images from 90 participants (45 each for type 2 diabetes and controls). FM-Net was comprised of a double Res-UNet CNN architecture, designed to compensate for the severe class imbalance and complex geometry of EAT. The first network accurately detected the region of interest (ROI) containing fat, and the second network performed targeted regional segmentation of the ROI. Performance of fat segmentation was improved by using fatmaps as input of FM-Net, to enhance fat features by combining out-of-phase, water, and fat phase images. Performance was evaluated using dice similarity coefficient (DSC) and 95% Hausdorff distance (HD95). Overall, FM-Net obtained a promising DSC of 86.3%, and a low HD95 of 3.11 mm, outperforming existing state-of-the-art methods. The proposed method enables automatic and accurate quantification of EAT from Dixon MRI data, which could enhance the understanding of the role of EAT in CVDs.