Diffusion models were initially developed for text-to-image generation and are now being utilized to generate high quality synthetic images. Preceded by GANs, diffusion models have shown impressive results using various evaluation metrics. However, commonly used metrics such as FID and IS are not suitable for determining whether diffusion models are simply reproducing the training images. Here we train StyleGAN and a diffusion model, using  BRATS20, BRATS21 and a chest x-ray pneumonia dataset, to synthesize brain MRI and chest x-ray images, and measure the correlation between the synthetic images and all training images. Our results show that diffusion models are more likely to memorize the training images, compared to StyleGAN, especially for small datasets and when using 2D slices from 3D volumes. Researchers should be careful when using diffusion models (and to some extent GANs) for medical imaging, if the final goal is to share the synthetic images. 

Spine Computed Tomography (SCT) is essential for identifying fractures, tumors and degenerative spine diseases, assisting medical practitioners in formulating an accurate diagnosis and treatment. One of the core element of SCT is reporting. The effectiveness of spine reporting is often limited by challenges such as an inadequate infrastructure and lack of experts. Automated SCT analysis has the potential to revolutionize spinal healthcare and improve patient outcomes. To achieve this objective, we proposed a framework for spine report generation that utilizes transformer architecture, trained on textual reports alongside the visual features extracted from the sagittal slices of the SCT volume. A foundation model is used to perform Knowledge Distillation (KD) alongside an encoder to ensure an optimal performance. The proposed framework is evaluated on the public dataset (VerSe20). The incorporation of KD results improved both the BERT and BLEU1 score on the dataset, from 0.7486 to 0.7522 and 0.6361 to 0.7291. Additionally, the proposed framework is evaluated using four different types of reports: original radiologist reports, reports without spine-level annotations, rephrased reports, and reports generated by ChatGPT-4o (ChatGPT). The evaluation without spine-level annotations demonstrates superior performance across most metrics, achieving the highest BLEU-1 and ROUGE-L scores, with a BLEU-1 of 0.9293 and a ROUGE-L score of 0.9297. In contrast, the other techniques achieved moderate scores across all metrics. Finally, experienced radiologists assessed the spine report and have given high rating to the original reports across all three criteria (completeness, conciseness and correctness), in comparison to the generated reports. This study’s findings suggest that omitting spine-level annotations can improve the quality of text generation.

Large annotated datasets are required for training deep learning models, but in medical imaging data sharing is often complicated due to ethics, anonymization and data protection legislation. Generative AI models, such as generative adversarial networks (GANs) and diffusion models, can today produce very realistic synthetic images, and can potentially facilitate data sharing. However, in order to share synthetic medical images it must first be demonstrated that they can be used for training different networks with acceptable performance. Here, we therefore comprehensively evaluate four GANs (progressive GAN, StyleGAN 1–3) and a diffusion model for the task of brain tumor segmentation (using two segmentation networks, U-Net and a Swin transformer). Our results show that segmentation networks trained on synthetic images reach Dice scores that are 80%–90% of Dice scores when training with real images, but that memorization of the training images can be a problem for diffusion models if the original dataset is too small. Our conclusion is that sharing synthetic medical images is a viable option to sharing real images, but that further work is required. The trained generative models and the generated synthetic images are shared on AIDA data hub.

Radiotherapy treatment planning takes substantial time, several hours per patient, as it involves manual segmentation of tumor and risk organs. Segmentation networks can be trained to automatically perform the segmentations, but typically require large annotated datasets for training. Sharing of sensitive data between hospitals, to create a larger dataset, is often difficult due to ethics and GDPR. Here we therefore demonstrate that federated learning is a solution to this problem, as then only the segmentation model is sent between each hospital and a global server. We export and preprocess brain tumor images from the oncology departments in Linköping and Lund, and use federated learning to train a global segmentation model using two different frameworks.

4D Flow Magnetic Resonance Imaging (4D Flow MRI) is a non-invasive measurement technique capable of quantifying blood flow across the cardiovascular system. While practical use is limited by spatial resolution and image noise, incorporation of trained super-resolution (SR) networks has potential to enhance image quality post-scan. However, these efforts have predominantly been restricted to narrowly defined cardiovascular domains, with limited exploration of how SR performance extends across the cardiovascular system; a task aggravated by contrasting hemodynamic conditions apparent across the cardiovasculature. The aim of our study was therefore to explore the generalizability of SR 4D Flow MRI using a combination of existing super-resolution base models, novel heterogeneous training sets, and dedicated ensemble learning techniques; the latter-most being effectively used for improved domain adaption in other domains or modalities, however, with no previous exploration in the setting of 4D Flow MRI. With synthetic training data generated across three disparate domains (cardiac, aortic, cerebrovascular), varying convolutional base and ensemble learners were evaluated as a function of domain and architecture, quantifying performance on both in-silico and acquired in-vivo data from the same three domains. Results show that both bagging and stacking ensembling enhance SR performance across domains, accurately predicting high-resolution velocities from low-resolution input data in-silico. Likewise, optimized networks successfully recover native resolution velocities from downsampled in-vivo data, as well as show qualitative potential in generating denoised SR-images from clinicallevel input data. In conclusion, our work presents a viable approach for generalized SR 4D Flow MRI, with the novel use of ensemble learning in the setting of advanced fullfield flow imaging extending utility across various clinical areas of interest.