liu.seSearch for publications in DiVA
Endre søk
Link to record
Permanent link

Direct link
Publikasjoner (10 av 28) Visa alla publikasjoner
Baravdish, G., Eilertsen, G., Jaroudi, R., Johansson, T., Malý, L. & Unger, J. (2024). A Hybrid Sobolev Gradient Method for Learning NODEs. Operations Research Forum, 5, Article ID 91.
Åpne denne publikasjonen i ny fane eller vindu >>A Hybrid Sobolev Gradient Method for Learning NODEs
Vise andre…
2024 (engelsk)Inngår i: Operations Research Forum, E-ISSN 2662-2556, Vol. 5, artikkel-id 91Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

The inverse problem of supervised reconstruction of depth-variable (time-dependent) parameters in ordinary differential equations is considered, with the typical application of finding weights of a neural ordinary differential equation (NODE) for a residual network with time continuous layers. The differential equation is treated as an abstract and isolated entity, termed a standalone NODE (sNODE), to facilitate for a wide range of applications. The proposed parameter reconstruction is performed by minimizing a cost functional covering a variety of loss functions and penalty terms. Regularization via penalty terms is incorporated to enhance ethical and trustworthy AI formulations. A nonlinear conjugate gradient mini-batch optimization scheme (NCG) is derived for the training having the benefit of including a sensitivity problem. The model (differential equation)-based approach is thus combined with a data-driven learning procedure. Mathematical properties are stated for the differential equation and the cost functional. The adjoint problem needed is derived together with the sensitivity problem. The sensitivity problem itself can estimate changes in the output under perturbation of the trained parameters. To preserve smoothness during the iterations, the Sobolev gradient is calculated and incorporated. Numerical results are included to validate the procedure for a NODE and synthetic datasets and compared with standard gradient approaches. For stability, using the sensitivity problem, a strategy for adversarial attacks is constructed, and it is shown that the given method with Sobolev gradients is more robust than standard approaches for parameter identification.

sted, utgiver, år, opplag, sider
Switzerland: Springer Nature, 2024
Emneord
Adversarial attacks, Deep learning, Inverse problems, Neural ordinary differential equations, Sobolev gradient
HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-208091 (URN)10.1007/s43069-024-00377-x (DOI)2-s2.0-85205866958 (Scopus ID)
Tilgjengelig fra: 2024-10-02 Laget: 2024-10-02 Sist oppdatert: 2024-12-12bibliografisk kontrollert
Ahmadian, A., Ding, Y., Eilertsen, G. & Lindsten, F. (2024). Unsupervised Novelty Detection in Pretrained Representation Space with Locally Adapted Likelihood Ratio. In: International Conference on Artificial Intelligence and Statistics 2024, Proceedings of Machine Learning Research: . Paper presented at 27th International Conference on Artificial Intelligence and Statistics (AISTATS), Valencia, SPAIN, MAY 02-04, 2024. , 238
Åpne denne publikasjonen i ny fane eller vindu >>Unsupervised Novelty Detection in Pretrained Representation Space with Locally Adapted Likelihood Ratio
2024 (engelsk)Inngår i: International Conference on Artificial Intelligence and Statistics 2024, Proceedings of Machine Learning Research, 2024, Vol. 238Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Detecting novelties given unlabeled examples of normal data is a challenging task in machine learning, particularly when the novel and normal categories are semantically close. Large deep models pretrained on massive datasets can provide a rich representation space in which the simple k-nearest neighbor distance works as a novelty measure. However, as we show in this paper, the basic k-NN method might be insufficient in this context due to ignoring the 'local geometry' of the distribution over representations as well as the impact of irrelevant 'background features'. To address this, we propose a fully unsupervised novelty detection approach that integrates the flexibility of k-NN with a locally adapted scaling of dimensions based on the 'neighbors of nearest neighbor' and computing a 'likelihood ratio' in pretrained (self-supervised) representation spaces. Our experiments with image data show the advantage of this method when off-the-shelf vision transformers (e.g., pretrained by DINO) are used as the feature extractor without any fine-tuning.

Serie
Proceedings of Machine Learning Research, ISSN 2640-3498
HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-203391 (URN)001221034002024 ()
Konferanse
27th International Conference on Artificial Intelligence and Statistics (AISTATS), Valencia, SPAIN, MAY 02-04, 2024
Tilgjengelig fra: 2024-05-08 Laget: 2024-05-08 Sist oppdatert: 2025-02-01
Knutsson, A., Unnebäck, J., Jönsson, D. & Eilertsen, G. (2023). CDF-Based Importance Sampling and Visualization for Neural Network Training. In: Thomas Höllt and Daniel Jönsson (Ed.), Eurographics Workshop on Visual Computing for Biology and Medicine: . Paper presented at VCBM 2023: Eurographics Workshop on Visual Computing for Biology and Medicine, Norrköping, Sweden, September 20 – 22, 2023.
Åpne denne publikasjonen i ny fane eller vindu >>CDF-Based Importance Sampling and Visualization for Neural Network Training
2023 (engelsk)Inngår i: Eurographics Workshop on Visual Computing for Biology and Medicine / [ed] Thomas Höllt and Daniel Jönsson, 2023Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Training a deep neural network is computationally expensive, but achieving the same network performance with less computation is possible if the training data is carefully chosen. However, selecting input samples during training is challenging as their true importance for the optimization is unknown. Furthermore, evaluation of the importance of individual samples must be computationally efficient and unbiased. In this paper, we present a new input data importance sampling strategy for reducing the training time of deep neural networks. We investigate different importance metrics that can be efficiently retrieved as they are available during training, i.e., the training loss and gradient norm. We found that choosing only samples with large loss or gradient norm, which are hard for the network to learn, is not optimal for the network performance. Instead, we introduce an importance sampling strategy that selects samples based on the cumulative distribution function of the loss and gradient norm, thereby making it more likely to choose hard samples while still including easy ones. The behavior of the proposed strategy is first analyzed on a synthetic dataset, and then evaluated in the application of classification of malignant cancer in digital pathology image patches. As pathology images contain many repetitive patterns, there could be significant gains in focusing on features that contribute stronger to the optimization. Finally, we show how the importance sampling process can be used to gain insights about the input data through visualization of samples that are found most or least useful for the training.

Serie
Eurographics Workshop on Visual Computing for Biomedicine, ISSN 2070-5778, E-ISSN 2070-5786
Emneord
Computing methodologies, Neural networks, Human-centered computing, Visualization techniques;
HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-199166 (URN)10.2312/vcbm.20231212 (DOI)978-3-03868-216-5 (ISBN)
Konferanse
VCBM 2023: Eurographics Workshop on Visual Computing for Biology and Medicine, Norrköping, Sweden, September 20 – 22, 2023
Forskningsfinansiär
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Merknad

The fulltext is published under Creative Common license Attribution 4.0https://creativecommons.org/licenses/by/4.0/

No changes have been made to the publication.

Tilgjengelig fra: 2023-11-13 Laget: 2023-11-13 Sist oppdatert: 2023-11-21
Pocevičiūtė, M., Eilertsen, G., Garvin, S. & Lundström, C. (2023). Detecting Domain Shift in Multiple Instance Learning for Digital Pathology Using Fréchet Domain Distance. In: Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor (Ed.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part V. Paper presented at MICCAI 2023, Vancouver, BC, Canada, October 8–12, 2023 (pp. 157-167). Springer, 14224
Åpne denne publikasjonen i ny fane eller vindu >>Detecting Domain Shift in Multiple Instance Learning for Digital Pathology Using Fréchet Domain Distance
2023 (engelsk)Inngår i: Medical Image Computing and Computer Assisted Intervention – MICCAI 2023: 26th International Conference, Vancouver, BC, Canada, October 8–12, 2023, Proceedings, Part V / [ed] Hayit Greenspan, Anant Madabhushi, Parvin Mousavi, Septimiu Salcudean, James Duncan, Tanveer Syeda-Mahmood, Russell Taylor, Springer, 2023, Vol. 14224, s. 157-167Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Multiple-instance learning (MIL) is an attractive approach for digital pathology applications as it reduces the costs related to data collection and labelling. However, it is not clear how sensitive MIL is to clinically realistic domain shifts, i.e., differences in data distribution that could negatively affect performance, and if already existing metrics for detecting domain shifts work well with these algorithms. We trained an attention-based MIL algorithm to classify whether a whole-slide image of a lymph node contains breast tumour metastases. The algorithm was evaluated on data from a hospital in a different country and various subsets of this data that correspond to different levels of domain shift. Our contributions include showing that MIL for digital pathology is affected by clinically realistic differences in data, evaluating which features from a MIL model are most suitable for detecting changes in performance, and proposing an unsupervised metric named Fréchet Domain Distance (FDD) for quantification of domain shifts. Shift measure performance was evaluated through the mean Pearson correlation to change in classification performance, where FDD achieved 0.70 on 10-fold cross-validation models. The baselines included Deep ensemble, Difference of Confidence, and Representation shift which resulted in 0.45, -0.29, and 0.56 mean Pearson correlation, respectively. FDD could be a valuable tool for care providers and vendors who need to verify if a MIL system is likely to perform reliably when implemented at a new site, without requiring any additional annotations from pathologists.

sted, utgiver, år, opplag, sider
Springer, 2023
Serie
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 14224
Emneord
Deep learning, domain shift detection, multiple instance learning, digital pathology
HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-199190 (URN)10.1007/978-3-031-43904-9_16 (DOI)001109633700016 ()2-s2.0-85174689282 (Scopus ID)9783031439032 (ISBN)9783031439049 (ISBN)
Konferanse
MICCAI 2023, Vancouver, BC, Canada, October 8–12, 2023
Forskningsfinansiär
Vinnova
Merknad

Funding: Swedish e-Science Research Center; VINNOVA; CENIIT career development program at Linkoping University; Wallenberg AI, WASP - Knut and Alice Wallenberg Foundation

Tilgjengelig fra: 2023-11-15 Laget: 2023-11-15 Sist oppdatert: 2025-02-09bibliografisk kontrollert
Hanji, P., Mantiuk, R. K., Eilertsen, G., Hajisharif, S. & Unger, J. (2022). Comparison of single image HDR reconstruction methods — the caveats of quality assessment. In: Munkhtsetseg Nandigjav,Niloy J. Mitra, Aaron Hertzmann (Ed.), SIGGRAPH '22: ACM SIGGRAPH 2022 Conference Proceedings: . Paper presented at SIGGRAPH '22: Special Interest Group on Computer Graphics and Interactive Techniques Conference Vancouver BC Canada August 7 - 11, 2022 (pp. 1-8). New York, NY, United States: Association for Computing Machinery (ACM), Article ID 1.
Åpne denne publikasjonen i ny fane eller vindu >>Comparison of single image HDR reconstruction methods — the caveats of quality assessment
Vise andre…
2022 (engelsk)Inngår i: SIGGRAPH '22: ACM SIGGRAPH 2022 Conference Proceedings / [ed] Munkhtsetseg Nandigjav,Niloy J. Mitra, Aaron Hertzmann, New York, NY, United States: Association for Computing Machinery (ACM), 2022, s. 1-8, artikkel-id 1Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

As the problem of reconstructing high dynamic range (HDR) imagesfrom a single exposure has attracted much research effort, it isessential to provide a robust protocol and clear guidelines on howto evaluate and compare new methods. In this work, we comparedsix recent single image HDR reconstruction (SI-HDR) methodsin a subjective image quality experiment on an HDR display. Wefound that only two methods produced results that are, on average,more preferred than the unprocessed single exposure images. Whenthe same methods are evaluated using image quality metrics, astypically done in papers, the metric predictions correlate poorlywith subjective quality scores. The main reason is a significant toneand color difference between the reference and reconstructed HDRimages. To improve the predictions of image quality metrics, we propose correcting for the inaccuracies of the estimated cameraresponse curve before computing quality values. We further analyzethe sources of prediction noise when evaluating SI-HDR methodsand demonstrate that existing metrics can reliably predict onlylarge quality differences.

sted, utgiver, år, opplag, sider
New York, NY, United States: Association for Computing Machinery (ACM), 2022
Emneord
High dynamic range, inverse problems, image quality metrics
HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-186401 (URN)10.1145/3528233.3530729 (DOI)9781450393379 (ISBN)
Konferanse
SIGGRAPH '22: Special Interest Group on Computer Graphics and Interactive Techniques Conference Vancouver BC Canada August 7 - 11, 2022
Merknad

Funding: This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement N° 725253–EyeCode)

Tilgjengelig fra: 2022-06-23 Laget: 2022-06-23 Sist oppdatert: 2024-08-26bibliografisk kontrollert
Stacke, K., Unger, J., Lundström, C. & Eilertsen, G. (2022). Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applications. The Journal of Machine Learning for Biomedical Imaging, 1, Article ID 023.
Åpne denne publikasjonen i ny fane eller vindu >>Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applications
2022 (engelsk)Inngår i: The Journal of Machine Learning for Biomedical Imaging, E-ISSN 2766-905X, Vol. 1, artikkel-id 023Artikkel i tidsskrift (Annet vitenskapelig) Published
Abstract [en]

Unsupervised learning has made substantial progress over the last few years, especially by means of contrastive self-supervised learning. The dominating dataset for benchmarking self-supervised learning has been ImageNet, for which recent methods are approaching the performance achieved by fully supervised training. The ImageNet dataset is however largely object-centric, and it is not clear yet what potential those methods have on widely different datasets and tasks that are not object-centric, such as in digital pathology.While self-supervised learning has started to be explored within this area with encouraging results, there is reason to look closer at how this setting differs from natural images and ImageNet. In this paper we make an in-depth analysis of contrastive learning for histopathology, pin-pointing how the contrastive objective will behave differently due to the characteristics of histopathology data. Using SimCLR and H&E stained images as a representative setting for contrastive self-supervised learning in histopathology, we bring forward a number of considerations, such as view generation for the contrastive objectiveand hyper-parameter tuning. In a large battery of experiments, we analyze how the downstream performance in tissue classification will be affected by these considerations. The results point to how contrastive learning can reduce the annotation effort within digital pathology, but that the specific dataset characteristics need to be considered. To take full advantage of the contrastive learning objective, different calibrations of view generation and hyper-parameters are required. Our results pave the way for realizing the full potential of self-supervised learning for histopathology applications. Code and trained models are available at https://github.com/k-stacke/ssl-pathology.

sted, utgiver, år, opplag, sider
Melba (The Journal of Machine Learning for Biomedical Imaging), 2022
HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-189163 (URN)
Tilgjengelig fra: 2022-10-12 Laget: 2022-10-12 Sist oppdatert: 2025-02-09
Eilertsen, G., Tsirikoglou, A., Lundström, C. & Unger, J. (2021). Ensembles of GANs for synthetic training data generation. In: : . Paper presented at ICLR 2021 workshop on Synthetic Data Generation: Quality, Privacy, Bias.
Åpne denne publikasjonen i ny fane eller vindu >>Ensembles of GANs for synthetic training data generation
2021 (engelsk)Konferansepaper, Oral presentation with published abstract (Fagfellevurdert)
Abstract [en]

Insufficient training data is a major bottleneck for most deep learning practices, not least in medical imaging where data is difficult to collect and publicly available datasets are scarce due to ethics and privacy. This work investigates the use of synthetic images, created by generative adversarial networks (GANs), as the only source of training data. We demonstrate that for this application, it is of great importance to make use of multiple GANs to improve the diversity of the generated data, i.e. to sufficiently cover the data distribution. While a single GAN can generate seemingly diverse image content, training on this data in most cases lead to severe over-fitting. We test the impact of ensembled GANs on synthetic 2D data as well as common image datasets (SVHN and CIFAR-10), and using both DCGANs and progressively growing GANs. As a specific use case, we focus on synthesizing digital pathology patches to provide anonymized training data.

HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-175900 (URN)
Konferanse
ICLR 2021 workshop on Synthetic Data Generation: Quality, Privacy, Bias
Forskningsfinansiär
Wallenberg AI, Autonomous Systems and Software Program (WASP)Vinnova, grant 2019-05144 and grant 2017-02447(AIDA)ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications
Tilgjengelig fra: 2021-05-26 Laget: 2021-05-26 Sist oppdatert: 2022-01-17
Tsirikoglou, A., Gladh, M., Sahlin, D., Eilertsen, G. & Unger, J. (2021). Generative inter-class transformations for imbalanced data weather classification. London Imaging Meeting, 2021, 16-20
Åpne denne publikasjonen i ny fane eller vindu >>Generative inter-class transformations for imbalanced data weather classification
Vise andre…
2021 (engelsk)Inngår i: London Imaging Meeting, E-ISSN 2694-118X, Vol. 2021, s. 16-20Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

This paper presents an evaluation of how data augmentation and inter-class transformations can be used to synthesize training data in low-data scenarios for single-image weather classification. In such scenarios, augmentations is a critical component, but there is a limit to how much improvements can be gained using classical augmentation strategies. Generative adversarial networks (GAN) have been demonstrated to generate impressive results, and have also been successful as a tool for data augmentation, but mostly for images of limited diversity, such as in medical applications. We investigate the possibilities in using generative augmentations for balancing a small weather classification dataset, where one class has a reduced number of images. We compare intra-class augmentations by means of classical transformations as well as noise-to-image GANs, to interclass augmentations where images from another class are transformed to the underrepresented class. The results show that it is possible to take advantage of GANs for inter-class augmentations to balance a small dataset for weather classification. This opens up for future work on GAN-based augmentations in scenarios where data is both diverse and scarce.

sted, utgiver, år, opplag, sider
Springfield, USA: Society for Imaging Science and Technology, 2021
HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-182334 (URN)10.2352/issn.2694-118X.2021.LIM-16 (DOI)
Merknad

Funding: This project was funded by Knut and Alice Wallenberg Foundation, Wallenberg Autonomous Systems and Software Program, the strategic research environment ELLIIT, and ‘AI for Climate Adaptation’ through VINNOVA grant 2020-03388.

Tilgjengelig fra: 2022-01-17 Laget: 2022-01-17 Sist oppdatert: 2025-02-07bibliografisk kontrollert
Stacke, K., Eilertsen, G., Unger, J. & Lundström, C. (2021). Measuring Domain Shift for Deep Learning in Histopathology. IEEE journal of biomedical and health informatics, 25(2), 325-336
Åpne denne publikasjonen i ny fane eller vindu >>Measuring Domain Shift for Deep Learning in Histopathology
2021 (engelsk)Inngår i: IEEE journal of biomedical and health informatics, ISSN 2168-2194, E-ISSN 2168-2208, Vol. 25, nr 2, s. 325-336Artikkel i tidsskrift (Fagfellevurdert) Published
Abstract [en]

The high capacity of neural networks allows fitting models to data with high precision, but makes generalization to unseen data a challenge. If a domain shift exists, i.e. differences in image statistics between training and test data, care needs to be taken to ensure reliable deployment in real-world scenarios. In digital pathology, domain shift can be manifested in differences between whole-slide images, introduced by for example differences in acquisition pipeline - between medical centers or over time. In order to harness the great potential presented by deep learning in histopathology, and ensure consistent model behavior, we need a deeper understanding of domain shift and its consequences, such that a model's predictions on new data can be trusted. This work focuses on the internal representation learned by trained convolutional neural networks, and shows how this can be used to formulate a novel measure - the representation shift - for quantifying the magnitude of model specific domain shift. We perform a study on domain shift in tumor classification of hematoxylin and eosin stained images, by considering different datasets, models, and techniques for preparing data in order to reduce the domain shift. The results show how the proposed measure has a high correlation with drop in performance when testing a model across a large number of different types of domain shifts, and how it improves on existing techniques for measuring data shift and uncertainty. The proposed measure can reveal how sensitive a model is to domain variations, and can be used to detect new data that a model will have problems generalizing to. We see techniques for measuring, understanding and overcoming the domain shift as a crucial step towards reliable use of deep learning in the future clinical pathology applications.

sted, utgiver, år, opplag, sider
IEEE, 2021
Emneord
deep learning, machine learning, domain shift, histopathology
HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-170816 (URN)10.1109/JBHI.2020.3032060 (DOI)000616310200003 ()
Merknad

Funding:  Wallenberg AI and Autonomous Systems and Software Program (WASP-AI); research environment ELLIIT; AIDA VinnovaVinnova [2017-02447]

Tilgjengelig fra: 2020-10-23 Laget: 2020-10-23 Sist oppdatert: 2023-04-03
Pocevičiūtė, M., Eilertsen, G. & Lundström, C. (2021). Unsupervised Anomaly Detection In Digital Pathology Using GANs. In: 2021 IEEE 18th International Symposium On Biomedical Imaging (ISBI): . Paper presented at 18th IEEE International Symposium on Biomedical Imaging (ISBI), Nice, FRANCE, apr 13-16, 2021 (pp. 1878-1882). Institute of Electrical and Electronics Engineers (IEEE)
Åpne denne publikasjonen i ny fane eller vindu >>Unsupervised Anomaly Detection In Digital Pathology Using GANs
2021 (engelsk)Inngår i: 2021 IEEE 18th International Symposium On Biomedical Imaging (ISBI), Institute of Electrical and Electronics Engineers (IEEE), 2021, s. 1878-1882Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Machine learning (ML) algorithms are optimized for the distribution represented by the training data. For outlier data, they often deliver predictions with equal confidence, even though these should not be trusted. In order to deploy ML-based digital pathology solutions in clinical practice, effective methods for detecting anomalous data are crucial to avoid incorrect decisions in the outlier scenario. We propose a new unsupervised learning approach for anomaly detection in histopathology data based on generative adversarial networks (GANs). Compared to the existing GAN-based methods that have been used in medical imaging, the proposed approach improves significantly on performance for pathology data. Our results indicate that histopathology imagery is substantially more complex than the data targeted by the previous methods. This complexity requires not only a more advanced GAN architecture but also an appropriate anomaly metric to capture the quality of the reconstructed images.

sted, utgiver, år, opplag, sider
Institute of Electrical and Electronics Engineers (IEEE), 2021
Serie
IEEE International Symposium on Biomedical Imaging, ISSN 1945-7928, E-ISSN 1945-8452
Emneord
digital pathology, deep learning, GAN, anomaly detection
HSV kategori
Identifikatorer
urn:nbn:se:liu:diva-178631 (URN)10.1109/ISBI48211.2021.9434141 (DOI)000786144100399 ()9781665429474 (ISBN)9781665412469 (ISBN)
Konferanse
18th IEEE International Symposium on Biomedical Imaging (ISBI), Nice, FRANCE, apr 13-16, 2021
Merknad

Funding: Swedish e-Science Research Center; VINNOVAVinnova [2017-02447]

Tilgjengelig fra: 2021-08-25 Laget: 2021-08-25 Sist oppdatert: 2023-09-28bibliografisk kontrollert
Organisasjoner
Identifikatorer
ORCID-id: ORCID iD iconorcid.org/0000-0002-9217-9997