liu.seSearch for publications in DiVA
Change search
Link to record
Permanent link

Direct link
BETA
Publications (6 of 6) Show all publications
Lindvall, M. & Molin, J. (2020). Designing for the Long Tail of Machine Learning.
Open this publication in new window or tab >>Designing for the Long Tail of Machine Learning
2020 (English)Manuscript (preprint) (Other academic)
Abstract [en]

Recent technical advances has made machine learning (ML) a promising component to include in end user facing systems. However, user experience (UX) practitioners face challenges in relating ML to existing user-centered design processes and how to navigate the possibilities and constraints of this design space. Drawing on our own experience, we characterize designing within this space as navigating trade-offs between data gathering, model development and designing valuable interactions for a given model performance. We suggest that the theoretical description of how machine learning performance scales with training data can guide designers in these trade-offs as well as having implications for prototyping. We exemplify the learning curve's usage by arguing that a useful pattern is to design an initial system in a bootstrap phase that aims to exploit the training effect of data collected at increasing orders of magnitude.

Keywords
digital design, interaction design, machine learning
National Category
Design Human Computer Interaction
Identifiers
urn:nbn:se:liu:diva-163530 (URN)
Note

Accepted for presentation in poster format for the ACM CHI'19 Workshop <Emerging Perspectives in Human-Centered Machine Learning>

Available from: 2020-02-07 Created: 2020-02-07 Last updated: 2020-02-19Bibliographically approved
Lindvall, M. & Molin, J. (2020). Verification Staircase: a Design Strategy for Actionable Explanations. In: Alison Smith-Renner and Styliani Kleanthous and Brian Lim and Tsvi Kuflik and Simone Stumpf and Jahna Otterbacher and Advait Sarkar and Casey Dugan and Avital Shulner Tal (Ed.), Proceedings of the Workshop on Explainable Smart Systems for Algorithmic Transparency in Emerging Technologies co-located with 25th International Conference on Intelligent User Interfaces (IUI 2020), Cagliari, Italy, March 17, 2020: . Paper presented at Workshop on Explainable Smart Systems for Algorithmic Transparency in Emerging Technologies co-located with 25th International Conference on Intelligent User Interfaces (IUI 2020), Cagliari, Italy, March 17–20, 2020. Aachen: CEUR-WS.org, 2582
Open this publication in new window or tab >>Verification Staircase: a Design Strategy for Actionable Explanations
2020 (English)In: Proceedings of the Workshop on Explainable Smart Systems for Algorithmic Transparency in Emerging Technologies co-located with 25th International Conference on Intelligent User Interfaces (IUI 2020), Cagliari, Italy, March 17, 2020 / [ed] Alison Smith-Renner and Styliani Kleanthous and Brian Lim and Tsvi Kuflik and Simone Stumpf and Jahna Otterbacher and Advait Sarkar and Casey Dugan and Avital Shulner Tal, Aachen: CEUR-WS.org , 2020, Vol. 2582Conference paper, Published paper (Refereed)
Abstract [en]

What if the trust in the output of a predictive model could be acted upon in richer ways than a simple binary decision of accept or reject? Designing assistive AI tools for medical specialists entails supporting a complex but safety-critical decision process. It is common that decisions in this domain can be decomposed to a combination of many smaller decisions. In this paper, we present Verification Staircase – a design strategy that can be used for such scenarios. The verification staircase is when multiple interactive assistive tools are combined to allow for a nuanced amount of automation to aid the user. This can support a wide range of prediction quality scenarios, spanning from unproblematic minor mistakes to misleading major failures. By presenting the information in a hierarchical way, the user is able to learn how underlying predictions are connected to overall case predictions, and over time, calibrate their trust so that they can choose the appropriate level of automatic support.

Place, publisher, year, edition, pages
Aachen: CEUR-WS.org, 2020
Series
CEUR Workshop Proceedings, ISSN 1613-0073
Keywords
digital design, interaction design, machine learning, artificial intelligence, explainable artificial intelligence
National Category
Design Human Computer Interaction Computer Sciences
Identifiers
urn:nbn:se:liu:diva-165763 (URN)
Conference
Workshop on Explainable Smart Systems for Algorithmic Transparency in Emerging Technologies co-located with 25th International Conference on Intelligent User Interfaces (IUI 2020), Cagliari, Italy, March 17–20, 2020
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Note

Accepted for oral presentation at Workshop on Explainable Smart Systems for Algorithmic Transparency in Emerging Technologies co-located with 25th International Conference on Intelligent User Interfaces (IUI 2020), Cagliari, Italy, March 17, 2020

Available from: 2020-05-20 Created: 2020-05-20 Last updated: 2020-05-27Bibliographically approved
Skoglund, K., Rose, J., Lindvall, M., Lundström, C. & Treanor, D. (2019). Annotations, ontologies, and whole slide images: Development of an annotated ontology-driven whole slide image library of normal and abnormal human tissue. Journal of Pathology Informatics, 10(22)
Open this publication in new window or tab >>Annotations, ontologies, and whole slide images: Development of an annotated ontology-driven whole slide image library of normal and abnormal human tissue
Show others...
2019 (English)In: Journal of Pathology Informatics, ISSN 2229-5089, E-ISSN 2153-3539, Vol. 10, no 22Article in journal (Refereed) Published
Abstract [en]

Objective: Digital pathology is today a widely used technology, and the digitalization of microscopic slides into whole slide images (WSIs) allows the use of machine learning algorithms as a tool in the diagnostic process. In recent years, “deep learning” algorithms for image analysis have been applied to digital pathology with great success. The training of these algorithms requires a large volume of high-quality images and image annotations. These large image collections are a potent source of information, and to use and share the information, standardization of the content through a consistent terminology is essential. The aim of this project was to develop a pilot dataset of exhaustive annotated WSI of normal and abnormal human tissue and link the annotations to appropriate ontological information. 

Materials and Methods: Several biomedical ontologies and controlled vocabularies were investigated with the aim of selecting the most suitable ontology for this project. The selection criteria required an ontology that covered anatomical locations, histological subcompartments, histopathologic diagnoses, histopathologic terms, and generic terms such as normal, abnormal, and artifact. WSIs of normal and abnormal tissue from 50 colon resections and 69 skin excisions, diagnosed 2015-2016 at the Department of Clinical Pathology in Linköping, were randomly collected. These images were manually and exhaustively annotated at the level of major subcompartments, including normal or abnormal findings and artifacts. 

Results: Systemized nomenclature of medicine clinical terms (SNOMED CT) was chosen, and the annotations were linked to its codes and terms. Two hundred WSI were collected and annotated, resulting in 17,497 annotations, covering a total area of 302.19 cm2, equivalent to 107,7 gigapixels. Ninety-five unique SNOMED CT codes were used. The time taken to annotate a WSI varied from 45 s to over 360 min, a total time of approximately 360 h. 

Conclusion: This work resulted in a dataset of 200 exhaustive annotated WSIs of normal and abnormal tissue from the colon and skin, and it has informed plans to build a comprehensive library of annotated WSIs. SNOMED CT was found to be the best ontology for annotation labeling. This project also demonstrates the need for future development of annotation tools in order to make the annotation process more efficient.

Place, publisher, year, edition, pages
Medknow Publications, 2019
Keywords
Annotation, digital pathology, image database, ontology, whole slide images
National Category
Radiology, Nuclear Medicine and Medical Imaging
Identifiers
urn:nbn:se:liu:diva-160146 (URN)10.4103/jpi.jpi_81_18 (DOI)
Available from: 2019-09-09 Created: 2019-09-09 Last updated: 2020-05-20Bibliographically approved
Jarkman, S., Lindvall, M., Hedlund, J., Treanor, D., Lundström, C. & van der Laak, J. (2019). Axillary lymph nodes in breast cancer cases.
Open this publication in new window or tab >>Axillary lymph nodes in breast cancer cases
Show others...
2019 (English)Data set, Primary data
Keywords
Pathology, Whole slide imaging, Breast, Lymph nodes, Cancer, Sentinel nodes, Immunohistochemical staining, cytokeratin, CKAE1/AE3
National Category
Radiology, Nuclear Medicine and Medical Imaging
Identifiers
urn:nbn:se:liu:diva-165757 (URN)10.23698/aida/brln (DOI)
Available from: 2020-05-19 Created: 2020-05-19 Last updated: 2020-05-26Bibliographically approved
Lindvall, M., Molin, J. & Löwgren, J. (2018). From machine learning to machine teaching: the importance of UX. interactions, 25(6), 52-57
Open this publication in new window or tab >>From machine learning to machine teaching: the importance of UX
2018 (English)In: interactions, ISSN 1072-5520, E-ISSN 1558-3449, Vol. 25, no 6, p. 52-57Article in journal (Refereed) Published
Place, publisher, year, edition, pages
New York: ACM Press, 2018
Keywords
digital design, interaction design, machine learning
National Category
Design Human Computer Interaction
Identifiers
urn:nbn:se:liu:diva-163529 (URN)10.1145/3282860 (DOI)2-s2.0-85056462250 (Scopus ID)
Available from: 2020-02-07 Created: 2020-02-07 Last updated: 2020-02-19Bibliographically approved
Tran Luciani, D., Lindvall, M. & Löwgren, J. (2018). Machine learning as a design material: a curated collection of exemplars for visual interaction. In: Philip Ekströmer, Simon Schütte and Johan Ölvander (Ed.), DS 91: Proceedings of NordDesign 2018, Linköping, Sweden, 14th - 17th August 2018: . Paper presented at The NordDesign 2018, Linköping, Sweden, 14th - 17th August 2018 (pp. 1-10).
Open this publication in new window or tab >>Machine learning as a design material: a curated collection of exemplars for visual interaction
2018 (English)In: DS 91: Proceedings of NordDesign 2018, Linköping, Sweden, 14th - 17th August 2018 / [ed] Philip Ekströmer, Simon Schütte and Johan Ölvander, 2018, p. 1-10Conference paper, Published paper (Refereed)
Abstract [en]

Although machine learning is not a new phenomenon, it has truly entered the spotlight in recent years. With growing expectations, we see a shift in focus from performance tuning to awareness of meaningful interaction and purpose. Interaction design and UX research is currently in a position to provide important and necessary knowledge contributions to the development of machine learning systems. Machine learning can be viewed as a design material that is arguably more unpredictable, emergent, and “alive” than traditional ones. These characteristics suggest practice-based work along the lines of research-through-design as a promising approach for machine learning system development research. Design researchers using a research-through-design approach agree that a created artefact carries knowledge, but there is no consensus on how such knowledge is best articulated and transferred within academic discourse. Knowledge contributions need to be abstracted from the particular to a higher level. We suggest curated collections, a variation of annotated portfolios, as a way to abstract and communicate intermediate-level knowledge that is suitable and useful for the research-through-design community. A curated collection presents thoughtfully selected and inter-related exemplars, articulating their salient traits. The insights collected in a curated collection can be used to inform future design in related design situations. This paper provides a curated collection addressing the fine-grained details of interaction with machine learning systems. The examples are drawn from highly visual interaction, predominantly in the domain of digital pathology. The collection of interaction examples is used to elicit a set of salient traits, including the preservation of visual context, rapid real-time refinement, leaving traces, and applying judicious automation. Finally, we show how this curated collection could inform the design of a future system in a different domain. The insights are applied to a case of interaction design to support air traffic controllers in their collaboration with future agentive systems

Series
NordDESIGN ; 2018
Keywords
digital design, interaction design, machine learning, curated collection
National Category
Design Human Computer Interaction Interaction Technologies Human Aspects of ICT Media and Communication Technology
Identifiers
urn:nbn:se:liu:diva-160675 (URN)9789176851852 (ISBN)
Conference
The NordDesign 2018, Linköping, Sweden, 14th - 17th August 2018
Available from: 2019-10-01 Created: 2019-10-01 Last updated: 2020-05-20Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-7014-8874

Search in DiVA

Show all publications