The impressive technical advances seen for machine learning algorithms in combination with the digitalization of medical images in the radiology and pathology departments show great promise in introducing powerful image analysis tools for image diagnostics. In particular, deep learning, a subfield within machine learning, has shown great success, advancing fields such as image classification and detection. However, these types of algorithms are only used to a very small extent in clinical practice. 

One reason is that the unique nature of radiology and pathology images and the clinical setting in which they are acquired poses challenges not seen in other image domains. Differences relate to capturing methods, as well as the image contents. In addition, these datasets are not only unique on a per-image basis but as a collective dataset. Characteristics such as size, class balance, and availability of annotated labels make creating robust and generalizable deep learning methods a challenge. 

This thesis investigates how deep learning models can be trained for applications in this domain, with particular focus on histopathology data. We investigate how domain shift between different scanners causes performance drop, and present ways of mitigating this. We also present a method to detect when domain shift occurs between different datasets. Another hurdle is the shortage of labeled data for medical applications, and this thesis looks at two different approaches to solving this problem. The first approach investigates how labeled data from one organ and cancer type can boost cancer classification in another organ where labeled data is scarce. The second approach looks at a specific type of unsupervised learning method, self-supervised learning, where the model is trained on unlabeled data. For both of these approaches, we present strategies to handle low-data regimes that may greatly increase the availability to build deep learning models for a wider range of applications. 

Furthermore, deep learning technology enables us to go beyond traditional medical domains, and combine the data from both radiology and pathology. This thesis presents a method for improved cancer characterization on contrast-enhanced CT by incorporating corresponding pathology data during training. The method shows the potential of im-proving future healthcare by intergraded diagnostics made possible by machine-learning technology.