liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Generalisation effects of predictive uncertainty estimation in deep learning for digital pathology
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Linköping University, Center for Medical Image Science and Visualization (CMIV).ORCID iD: 0000-0002-8734-6500
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Linköping University, Center for Medical Image Science and Visualization (CMIV).ORCID iD: 0000-0002-9217-9997
Linköping University, Department of Biomedical and Clinical Sciences, Division of Neurobiology. Linköping University, Faculty of Medicine and Health Sciences. Region Östergötland, Center for Diagnostics, Clinical pathology. Linköping University, Center for Medical Image Science and Visualization (CMIV).
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Linköping University, Center for Medical Image Science and Visualization (CMIV). Sectra AB, Linkoping, Sweden.ORCID iD: 0000-0002-9368-0177
2022 (English)In: Scientific Reports, E-ISSN 2045-2322, Vol. 12, no 1, article id 8329Article in journal (Refereed) Published
Abstract [en]

Deep learning (DL) has shown great potential in digital pathology applications. The robustness of a diagnostic DL-based solution is essential for safe clinical deployment. In this work we evaluate if adding uncertainty estimates for DL predictions in digital pathology could result in increased value for the clinical applications, by boosting the general predictive performance or by detecting mispredictions. We compare the effectiveness of model-integrated methods (MC dropout and Deep ensembles) with a model-agnostic approach (Test time augmentation, TTA). Moreover, four uncertainty metrics are compared. Our experiments focus on two domain shift scenarios: a shift to a different medical center and to an underrepresented subtype of cancer. Our results show that uncertainty estimates increase reliability by reducing a models sensitivity to classification threshold selection as well as by detecting between 70 and 90% of the mispredictions done by the model. Overall, the deep ensembles method achieved the best performance closely followed by TTA.

Place, publisher, year, edition, pages
Nature Portfolio , 2022. Vol. 12, no 1, article id 8329
National Category
Bioinformatics (Computational Biology)
Identifiers
URN: urn:nbn:se:liu:diva-185374DOI: 10.1038/s41598-022-11826-0ISI: 000797636300082PubMedID: 35585087OAI: oai:DiVA.org:liu-185374DiVA, id: diva2:1662375
Note

Funding Agencies|Swedish e-Science Research Center; VINNOVA [2017-02447]

Available from: 2022-05-31 Created: 2022-05-31 Last updated: 2023-09-28
In thesis
1. Generalisation and reliability of deep learning for digital pathology in a clinical setting
Open this publication in new window or tab >>Generalisation and reliability of deep learning for digital pathology in a clinical setting
2023 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Deep learning (DL) is a subfield of artificial intelligence (AI) focused on developing algorithms that learn from data to perform some tasks that can aid humans in their daily life or work assignments. Research demonstrates the potential of DL in supporting pathologists with routine tasks like detecting breast cancer metastases and grading prostate cancer. However, a widespread adoption of DL technology in pathology labs has been slow for several reasons. DL models often exhibit performance variations across medical centres, patient subgroups, and even within the same centre over time. While collecting more data and retraining the algorithms seems like a straightforward solution, it is a costly and time-consuming process. Moreover, retraining DL systems with regulatory approvals is complex due to existing regulations. Another limitation of DL models is their inability to provide confidence estimates for predictions, leaving users in the dark about their reliability. Finally, establishing a close collaboration between the research community, vendors, and pathology labs is crucial for producing effective DL systems for patient care. However, this collaboration faces challenges like miscommunication, misalignment of goals, and misunderstanding priorities.

This thesis presents various approaches that could tackle the generalisation and reliability challenges faced by diagnostic DL systems for digital pathology with a strong emphasis on the clinical needs. To address the generalisation issues, an unsupervised approach to quantify expected changes in a model’s performance between two datasets is proposed. This approach can serve as an initial validation step before deploying diagnostic DL systems in clinical practice, reducing annotation costs. Additionally, an unsupervised framework based on generative models is proposed to identify substantially different inputs, known as out-of-distribution (OOD) samples. Detecting OOD samples plays a crucial role in enhancing the reliability of DL algorithms. Furthermore, several studies are conducted to explore what benefits uncertainty estimation could bring. Firstly, various uncertainty estimation approaches are extensively evaluated, focusing on identifying incorrect predictions and generalisability issues between medical centres and specific patient groups. In addition, the results reveal that combining uncertainty estimation methods with DL outputs leads to a more robust classification score, enhancing the overall performance and reliability of the classification process. Another study demonstrates that spatial uncertainty aggregation improves the effectiveness of uncertainty estimation in tumour segmentation tasks. This is evaluated on the detection of false negatives which may reduce the risk of missing tumour cells. Finally, the clinical prerequisites for developing and validating diagnostic DL systems for digital pathology are discussed, along with an overview of explainable AI techniques.

In conclusion, multiple approaches to facilitate the adoption of DL systems in clinical practice, addressing reliability, generalisability, and clinical needs aspects are discussed in this thesis. I believe that the extensive efforts in the research community will have a positive impact on the development, validation, and deployment of DL systems in digital pathology labs, empowering pathologists with trustworthy AI tools.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2023. p. 55
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 2337
Keywords
Deep learning, Digital pathology, Generalisation, Uncertainty estimation, Anomaly detection, Data distribution shift
National Category
Biomedical Laboratory Science/Technology
Identifiers
urn:nbn:se:liu:diva-198154 (URN)10.3384/9789180753005 (DOI)9789180752992 (ISBN)9789180753005 (ISBN)
Public defence
2023-11-03, Wrannesalen, Center for Medical Image Science and Visualization, Linköping University Hospital, Linköping, 09:15 (English)
Opponent
Supervisors
Available from: 2023-09-28 Created: 2023-09-28 Last updated: 2023-09-28Bibliographically approved

Open Access in DiVA

fulltext(2389 kB)247 downloads
File information
File name FULLTEXT01.pdfFile size 2389 kBChecksum SHA-512
e22ef250f694b6d036afd9a17d81af7aa9a35acd645cfc2b7a460b58aabc19baa4250f64d2837ef86b644f01dbc8b139b5e9698f5b092c980f4ac9a113d823b0
Type fulltextMimetype application/pdf

Other links

Publisher's full textPubMed

Search in DiVA

By author/editor
Poceviciute, MildaEilertsen, GabrielJarkman, SofiaLundström, Claes
By organisation
Media and Information TechnologyFaculty of Science & EngineeringCenter for Medical Image Science and Visualization (CMIV)Division of NeurobiologyFaculty of Medicine and Health SciencesClinical pathology
In the same journal
Scientific Reports
Bioinformatics (Computational Biology)

Search outside of DiVA

GoogleGoogle Scholar
Total: 248 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
pubmed
urn-nbn

Altmetric score

doi
pubmed
urn-nbn
Total: 451 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf