liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Machine learning as a design material: a curated collection of exemplars for visual interaction
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. Linköping University, Center for Medical Image Science and Visualization (CMIV). Sectra AB, Sweden.ORCID iD: 0000-0002-7014-8874
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0001-5678-6565
2018 (English)In: DS 91: Proceedings of NordDesign 2018, Linköping, Sweden, 14th - 17th August 2018 / [ed] Philip Ekströmer, Simon Schütte and Johan Ölvander, Brandes & Apsel Verlag, 2018, p. 1-10Conference paper, Published paper (Refereed)
Abstract [en]

Although machine learning is not a new phenomenon, it has truly entered the spotlight in recent years. With growing expectations, we see a shift in focus from performance tuning to awareness of meaningful interaction and purpose. Interaction design and UX research is currently in a position to provide important and necessary knowledge contributions to the development of machine learning systems. Machine learning can be viewed as a design material that is arguably more unpredictable, emergent, and “alive” than traditional ones. These characteristics suggest practice-based work along the lines of research-through-design as a promising approach for machine learning system development research. Design researchers using a research-through-design approach agree that a created artefact carries knowledge, but there is no consensus on how such knowledge is best articulated and transferred within academic discourse. Knowledge contributions need to be abstracted from the particular to a higher level. We suggest curated collections, a variation of annotated portfolios, as a way to abstract and communicate intermediate-level knowledge that is suitable and useful for the research-through-design community. A curated collection presents thoughtfully selected and inter-related exemplars, articulating their salient traits. The insights collected in a curated collection can be used to inform future design in related design situations. This paper provides a curated collection addressing the fine-grained details of interaction with machine learning systems. The examples are drawn from highly visual interaction, predominantly in the domain of digital pathology. The collection of interaction examples is used to elicit a set of salient traits, including the preservation of visual context, rapid real-time refinement, leaving traces, and applying judicious automation. Finally, we show how this curated collection could inform the design of a future system in a different domain. The insights are applied to a case of interaction design to support air traffic controllers in their collaboration with future agentive systems

Place, publisher, year, edition, pages
Brandes & Apsel Verlag, 2018. p. 1-10
Series
NordDESIGN ; 2018
Keywords [en]
digital design, interaction design, machine learning, curated collection
National Category
Design Human Computer Interaction Interaction Technologies Human Aspects of ICT Media and Communication Technology
Identifiers
URN: urn:nbn:se:liu:diva-160675ISBN: 9789176851852 (print)OAI: oai:DiVA.org:liu-160675DiVA, id: diva2:1356180
Conference
The NordDesign 2018, Linköping, Sweden, 14th - 17th August 2018
Available from: 2019-10-01 Created: 2019-10-01 Last updated: 2022-12-08Bibliographically approved
In thesis
1. Designing for sketching to support concept exploration
Open this publication in new window or tab >>Designing for sketching to support concept exploration
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Sketching is a way of exploring early concepts through the act of externalization in a suitable material with the aid of a suitable tool. One could use paper and sketch with a pencil or go digital and sketch with code. What is appropriate to choose depends on the situation and on the skillset of the person who is going to sketch. When sketching is done successfully, the externalization can “speak back” and thus engage the sketcher, and others, in a conversation leading to a better understanding of the sketched concepts as well as new concept ideas. This is thoroughly documented in literature – a typical example would be an architect sketching a site plan on a flat piece of paper and being able to read into the possible movements in the third-dimensional space. Sketching generally works like this in familiar, that is, idiomatic situations for experienced sketchers. In unfamiliar or non-idiomatic situations, existing sketching tools are inadequate for expressing and exploring early concepts. For novice sketchers, with limited sketching literacy, even attempting to sketch in an idiomatic situation can be challenging.

Through three cases, I design for concept exploration by enabling sketching to understand how this can be done in new situations. The first case deals with expert sketchers exploring non-idiomatic situations: professional creatives working with fulldome format for visual communication. The second case deals with novice sketchers exploring non-idiomatic situations: design students working with virtual reality. The third case deals with novice sketchers exploring idiomatic situations: air traffic controllers working with finding alternative routes for aircraft in the airspace with automation support.

I take a constructive design approach by making design examples and reflecting during and after the process. With the help of the design examples, I engage domain experts through participatory co-design workshops and elicit insights in order to inspire further design work. What I learn through this dynamic making-workshopping-and-reflecting process forms the foundation of the knowledge contribution. It is presented here as three design tactics on how sketching could be like to support concept exploration: 1) be responsive, 2) emulate salient material properties, and 3) be lightweight.

Abstract [sv]

Att skissa är ett sätt att utforska tidiga koncept. Det görs vanligtvis genom att externalisera koncepten i ett lämpligt material med hjälp av ett lämpligt verktyg. Det går likväl att skissa med papper och penna som att skissa digitalt med hjälp av kod. Vilket material eller verktyg som är att föredra beror på situationen och skickligheten hos personen som ska skissa. När skissning sker framgångsrikt kan externaliseringen “prata tillbaka” och på så sätt engagera skissaren, och andra, i en konversation som leder både till bättre förståelse för koncepten och även till nya konceptförslag. Det här finns utförligt beskrivet i litteraturen – ett typexempel är när en arkitekt skissar en byggnadskonstruktion på ett plant papper och utifrån det kan se hur möjliga rörelser kan te sig i en tredimensionell rymd. Skissning funkar generellt sett på det här sättet i alla fall i välbekanta, idiomatiska situationer för erfarna skissare. Däremot i obekanta, icke-idiomatiska situationer, är befintliga skissningsverktyg otillräckliga för att kunna användas till att uttrycka och utforska tidiga koncept. För novisa skissare, med begränsad skiss-kunnighet, är det dessutom utmanande att skissa även i en idiomatisk situation.

I tre fallstudier, designar jag för konceptutforskning för att förstå hur det kan ske i nya situationer genom att möjliggöra för skissning. Den första fallstudien handlar om erfarna skissare som utforskar icke-idiomatiska situationer: professionella kreatörer som jobbar med domformat för visuell kommunikation. Den andra fallstudien handlar om novisa skissare som utforskar icke-idiomatiska situationer: designstudenter som jobbar med virtuell verklighet. Den tredje fallstudien handlar om novisa skissare som utforskar idiomatiska situationer: flygledare som jobbar med att hitta alternativa rutter för flygmaskiner i luften med automationsstöd.

Jag tar en konstruktiv designansats genom ett skapande av designexempel och reflektion under och efter processen. Med hjälp av designexemplen engagerar jag domänexperter i deltagande co-designworkshops och fram insikter som inspirerar fortsatt designarbete. Det jag lär mig under den här dynamiska processen av skapande, workshopande och reflekterande, formar grunden till kunskapsbidraget. Bidraget presenteras här som tre designtaktiker om hur skissning kan vara för att stödja konceptutforskande: 1) vara responsiv, 2) emulera centrala materialegenskaper, och 3) vara lättviktig.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2019. p. 133
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 2013
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:liu:diva-161712 (URN)10.3384/diss.diva-161712 (DOI)9789175190099 (ISBN)
Public defence
2019-12-20, K3, Kåkenhus, Campus Norrköping, Norrköping, 09:00 (English)
Opponent
Supervisors
Available from: 2019-11-07 Created: 2019-11-07 Last updated: 2022-12-08Bibliographically approved
2. Designing with Machine Learning in Digital Pathology: Augmenting Medical Specialists through Interaction Design
Open this publication in new window or tab >>Designing with Machine Learning in Digital Pathology: Augmenting Medical Specialists through Interaction Design
2021 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Recent advancements in machine learning (ML) have led to a dramatic increase in AI capabilities for medical diagnostic tasks. Despite technical advances, developers of predictive AI models struggle to integrate their work into routine clinical workflows. Inefficient human-AI interactions, poor sociotechnical fit and a lack of interactive strategies for dealing with the imperfect nature of predictions are known factors contributing to this lack of adoption.

User-centred design methods are typically aimed at discovering and realising desirable qualities in use, pragmatically oriented around finding solutions despite the limitations of material- and human resources. However, existing methods often rely on designers possessing knowledge of suitable interactive metaphors and idioms, as well as skills in evaluating ideas through low-fidelity prototyping and rapid iteration methods—all of which are challenged by the data-driven nature of machine learning and the unpredictable outputs from AI models.

Using a constructive design research approach, my work explores how we might design systems with AI components that aid clinical decision-making in a human-centred and iterative fashion. Findings are derived from experiments and experiences from four exploratory projects conducted in collaboration with professional physicians, all aiming to probe this design space by producing novel interactive systems for or with ML components.

Contributions include identifying practical and theoretical design challenges, suggesting novel interaction strategies for human-AI collaboration, framing ML competence for designers and presenting empirical descriptions of conducted design processes. Specifically, this compilation thesis contains three works that address effective human-machine teaching and two works that address the challenge of designing interactions that afford successful decision-making despite the uncertainty and imperfections inherent in machine predictions.

Finally, two works directly address design-researchers working with ML, arguing for a systematic approach to increase the repertoire available for theoretical annotation and understanding of the properties of ML as a designerly material.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2021
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 2157
National Category
Design Human Computer Interaction
Identifiers
urn:nbn:se:liu:diva-176117 (URN)10.3384/diss.diva-176117 (DOI)978-91-7929-604-9 (ISBN)
Public defence
2021-09-23, K3, Kåkenhus, Campus Norrköping, Norrköping, 09:00 (English)
Opponent
Supervisors
Available from: 2021-08-30 Created: 2021-06-07 Last updated: 2022-12-08Bibliographically approved

Open Access in DiVA

No full text in DiVA

Authority records

Tran Luciani, DanweiLindvall, MartinLöwgren, Jonas

Search in DiVA

By author/editor
Tran Luciani, DanweiLindvall, MartinLöwgren, Jonas
By organisation
Media and Information TechnologyFaculty of Science & EngineeringCenter for Medical Image Science and Visualization (CMIV)
DesignHuman Computer InteractionInteraction TechnologiesHuman Aspects of ICTMedia and Communication Technology

Search outside of DiVA

GoogleGoogle Scholar

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 541 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf