Recent advancements in machine learning (ML) have led to a dramatic increase in AI capabilities for medical diagnostic tasks. Despite technical advances, developers of predictive AI models struggle to integrate their work into routine clinical workflows. Inefficient human-AI interactions, poor sociotechnical fit and a lack of interactive strategies for dealing with the imperfect nature of predictions are known factors contributing to this lack of adoption.

User-centred design methods are typically aimed at discovering and realising desirable qualities in use, pragmatically oriented around finding solutions despite the limitations of material- and human resources. However, existing methods often rely on designers possessing knowledge of suitable interactive metaphors and idioms, as well as skills in evaluating ideas through low-fidelity prototyping and rapid iteration methods—all of which are challenged by the data-driven nature of machine learning and the unpredictable outputs from AI models.

Using a constructive design research approach, my work explores how we might design systems with AI components that aid clinical decision-making in a human-centred and iterative fashion. Findings are derived from experiments and experiences from four exploratory projects conducted in collaboration with professional physicians, all aiming to probe this design space by producing novel interactive systems for or with ML components.

Contributions include identifying practical and theoretical design challenges, suggesting novel interaction strategies for human-AI collaboration, framing ML competence for designers and presenting empirical descriptions of conducted design processes. Specifically, this compilation thesis contains three works that address effective human-machine teaching and two works that address the challenge of designing interactions that afford successful decision-making despite the uncertainty and imperfections inherent in machine predictions.

Finally, two works directly address design-researchers working with ML, arguing for a systematic approach to increase the repertoire available for theoretical annotation and understanding of the properties of ML as a designerly material.

Breast cancer is the most common cancer among women in Sweden, and pathology is central to diagnosis and treatment planning. Advances in digital pathology, using high-resolution whole-slide images, (WSI), have  enabled the use of AI-based (artificial intelligence) image analysis tools that improves diagnostic efficiency and reproducibility. However, responsible implementation requires careful attention to clinical accountability, data governance and human oversight.

This thesis presents a multimodal evaluation framework and contributes to technical and medical insights to the field. In the first study a research database was constructed to support generalizable AI development, compromising six annotated imaging collections, comprising 754 WSIs and 24,043 pathology annotations, 110 radiology cases and 397 lesion annotations.

One study evaluated a human-in-the-loop (HITL) workflow, when pathologist assess cellproliferation as expressed by Ki-67 from an AI result. Even though AI showed reduced cell-level performance on local data, (F1 score 0.68 compared to 0.83), status agreement for visual estimation of Ki-67 performed significantly worse (Cohen’s κ 0.62) than digital image analysis (κ 0.84)  and HITL (κ 0.76). HITL reduced variability of the Ki-67 error and mitigated key limitations such as tumour heterogeneity, misidentification, and staining variability, while highlighting risks from user handling errors.

Subsequent studies introduced Feature Enhancing Zoom (FEZ), a visualization technique that amplifies stain patterns at low magnification. In a study with eight pathologists, FEZ improved task efficiency by 15% without compromising accuracy. A usability study with 16 pathologists confirmed high ratings, especially for stains requiring small object identification.

Finally, a methodology was evaluated to mitigate domain restraints during clinical implementation of a pretrained AI model for detecting metastases in lymph nodes. A locally curated dataset of 396 cases (4,462 WSIs) was used, with slides labelled by surgical procedure and lesion presence. Results showed that surgical procedure affects model performance, and retraining significantly improved generalization and thus reducing false positive predictions.

In summary, this thesis demonstrates how AI and digital pathology can be integrated into clinical workflows to enhance diagnostic precision.