Brain tumors are the most common solid tumors in children and young adults, but the scarcity of large histopathology datasets has limited the application of computational pathology in this group. This study implements two weakly supervised multiple-instance learning (MIL) approaches on patch features obtained from state-of-the-art histology-specific foundation models to classify pediatric brain tumors in hematoxylin and eosin whole slide images (WSIs) from a multi-center Swedish cohort. WSIs from 540 subjects (age 8.5 ± 4.9 years) diagnosed with brain tumors were gathered from the six Swedish university hospitals. Instance (patch)-level features were obtained from WSIs using three pre-trained feature extractors: ResNet50, UNI, and CONCH. Instances were aggregated using attention-based MIL (ABMIL) or clustering-constrained attention MIL (CLAM) for patient-level classification. Models were evaluated on three classification tasks based on the hierarchical classification of pediatric brain tumors: tumor category, family, and type. Model generalization was assessed by training on data from two of the centers and testing on data from four other centers. Model interpretability was evaluated through attention mapping. The highest classification performance was achieved using UNI features and ABMIL aggregation, with Matthew's correlation coefficient of 0.76 ± 0.04, 0.63 ± 0.04, and 0.60 ± 0.05 for tumor category, family, and type classification, respectively. When evaluating generalization, models utilizing UNI and CONCH features outperformed those using ResNet50. However, the drop in performance from the in-site to out-of-site testing was similar across feature extractors. These results show the potential of state-of-the-art computational pathology methods in diagnosing pediatric brain tumors at different hierarchical levels with fair generalizability on a multi-center national dataset.

Purpose: To implement and evaluate deep learning-based methods for the classification of pediatric brain tumors in MR data.

Materials and methods: A subset of the “Children’s Brain Tumor Network” dataset was retrospectively used (n=178 subjects, female=72, male=102, NA=4, age-range [0.01, 36.49] years) with tumor types being low-grade astrocytoma (n=84), ependymoma (n=32), and medulloblastoma (n=62). T1w post-contrast (n=94 subjects), T2w (n=160 subjects), and ADC (n=66 subjects) MR sequences were used separately. Two deep-learning models were trained on transversal slices showing tumor. Joint fusion was implemented to combine image and age data, and two pre-training paradigms were utilized. Model explainability was investigated using gradient-weighted class activation mapping (Grad-CAM), and the learned feature space was visualized using principal component analysis (PCA).

Results: The highest tumor-type classification performance was achieved when using a vision transformer model pre-trained on ImageNet and fine-tuned on ADC images with age fusion (MCC: 0.77 ± 0.14 Accuracy: 0.87 ± 0.08), followed by models trained on T2w (MCC: 0.58 ± 0.11, Accuracy: 0.73 ± 0.08) and T1w post-contrast (MCC: 0.41 ± 0.11, Accuracy: 0.62 ± 0.08) data. Age fusion marginally improved the model’s performance. Both model architectures performed similarly across the experiments, with no differences between the pre-training strategies. Grad-CAMs showed that the models’ attention focused on the brain region. PCA of the feature space showed greater separation of the tumor-type clusters when using contrastive pre-training.

Conclusion: Classification of pediatric brain tumors on MR-images could be accomplished using deep learning, with the top-performing model being trained on ADC data, which is used by radiologists for the clinical classification of these tumors.

Medical imaging is one of the cornerstones of clinical diagnosis, providing insights into the anatomy and physiology of organs and tissues for screening, initial diagnosis, treatment planning, and follow-up. Utilizing both invasive and non-invasive techniques, medical imaging employs various contrast mechanisms to capture details of the tissue structure and the functionality of biological systems at different spatial and temporal resolutions, and dimensionalities. The ever-growing volume of medical image data driven by screening programs, digitalization, and the push towards precision medicine has highlighted the need for automatic image analysis methods to reduce the workload of healthcare personnel in reviewing these images.

Deep learning (DL), a subset of artificial intelligence (AI), comprises of methods that learn representations from data to perform various predictive tasks. Although DL was introduced in the mid-1960s, it has only been successfully applied for computer vision tasks in the past two decades, becoming the standard method for natural image processing. Additionally, the versatility of DL in processing data from diverse sources (such as speech, text, and climate) has encouraged its application in the medical domain as well.

This thesis explores the application of DL-based methods for medical image analysis, focusing on cancer diagnosis at various treatment planning stages, including preoperative, intraoperative, and postoperative procedures. Methods were developed and applied to three medical imaging modalities: optical coherence tomography (OCT) for intraoperative diagnosis, magnetic resonance imaging (MRI) for pre-operative diagnosis and radiotherapy treatment planning, and histopathology whole-slide images (WSI) for postoperative final diagnosis, addressing tasks such as detection, semantic segmentation, and classification for thyroid diseases and pediatric and adult brain tumors.

In summary, the outcomes of this thesis highlight the potential of deep learning-based methods for medical image analysis in the context of cancer diagnosis. These works demonstrate the versatility of deep learning in processing medical images from various sources and at different spatial resolutions and dimensionalities. Appropriate dataset curation, method validation and interpretation, and translational research are needed to promote the integration of deep learning-powered tools in the clinic.

This study explores the application of computational pathology to analyze pediatric brain tumors utilizing hematoxylin and eosin (H&E) and immunohistochemistry (IHC) whole slide images (WSIs). Experiments were conducted on H&E images for predicting tumor diagnosis and fusing them with unregistered IHC images to investigate potential improvements. Patch features were extracted using UNI, a vision transformer (ViT) model trained on H&E data, and whole slide classification was achieved using the attention-based multiple instance learning CLAM framework. In the astrocytoma tumor classification, early fusion of the H&E and IHC significantly improved the differentiation between tumor grades (balanced accuracy: 0.82±0.05vs 0.84 ± 0.05). In the multiclass classification, H&E images alone had a balanced accuracy of 0.79 ± 0.03 without any improvement obtained when fused with IHC. The findings highlight the potential of using multi-stain fusion to advance the diagnosis of pediatric brain tumors, however, further fusion methods should be investigated.

Deep learning models have achieved prominent performance in digital pathology, with the potential to provide healthcare professionals with accurate decision-making assistance in their workflow. In this study, ViT and CNN models were implemented and compared for patch-level classification of four major glioblastoma tissue structures in histology images.

A subset of the IvyGAP dataset (41 subjects, 123 images) was used, stain-normalised and patches of size 256x256 pixels were extracted. A per-subject split approach was applied to obtain training, validation and testing sets. Three models were implemented, a ViT and a CNN trained from scratch, and a ViT pre-trained on a different brain tumour histology dataset. The models' performance was assessed using a range of metrics, including accuracy and Matthew's correlation coefficient (MCC). In addition, calibration experiments were conducted and evaluated to align the models with the ground truth, utilising the temperature scaling technique. The models' uncertainty was estimated using the Monte Carlo dropout method. Lastly, the models were compared using the Wilcoxon signed-rank statistical significance test with Bonferroni correction.

Among the models, the scratch-trained ViT obtained the highest test accuracy of 67% and an MCC of 0.45. The scratch-trained CNN reached a test accuracy of 49% and an MCC of 0.15, and the pre-trained ViT only achieved a test accuracy of 28% and an MCC of 0.034. Comparing the reliability graphs and metrics before and after applying temperature scaling, the subsequent experiments proceeded with the uncalibrated ViTs and the calibrated CNN. The calibrated CNN demonstrated moderate to high uncertainty across classes, and the ViTs had an overall high uncertainty. Statistically, there was no difference among the models at a significance level of 0.017. 

In conclusion, the scratch-trained ViT model considerably outperformed the scratch-trained CNN and the pre-trained ViT in classification. However, there was no statistically significant difference among the models.

The infiltrative nature of malignant gliomas results in active tumor spreading into the peritumoral edema, which is not visible in conventional magnetic resonance imaging (cMRI) even after contrast injection. MR relaxometry (qMRI) measures relaxation rates dependent on tissue properties, and can offer additional contrast mechanisms to highlight the non-enhancing infiltrative tumor. To investigate if qMRI data provides additional information compared to cMRI sequences when considering deep learning-based brain tumor detection and segmentation, preoperative conventional (T1-w per- and post-contrast, T2-w and FLAIR) and quantitative (pre- and post-contrast R1, R2 and proton density) MR data was obtained from 23 patients with typical radiological findings suggestive of a high-grade malignant glioma. 2D deep learning models were trained on transversal slices (n=528) for tumor detection and segmentation using either conventional or quantitative data. Moreover, trends in quantitative R1 and R2 rates of regions identified as relevant for tumor detection by model explainability methods were qualitatively analyzed. Tumor detection and segmentation performance for models trained with a combination of qMRI pre- and post-contrast was the highest (detection MCC=0.72, segmentation DSC=0.90), however, the difference compared to cMRI was not statistically significant. Overall analysis of the relevant regions identified using model explainability showed no differences between models trained on cMRI or qMRI. When looking at the individual cases, relaxation rates of brain regions outside the annotation and identified as relevant for tumor detection exhibited changes after contrast injection similar to region inside the annotation in the majority of cases. In conclusion, models trained on qMRI data obtained similar detection and segmentation performance to those trained on cMRI data, with the advantage of quantitatively measuring brain tissue properties within similar scan time. When considering individual patients, the analysis of relaxation rates of regions identified by model explainability suggests the presence of infiltrative tumor outside the tumor cMRI-based annotation.