liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Designing Human-Swarm Interaction Systems
Linköping University, Department of Computer and Information Science, Human-Centered Systems. Linköping University, Faculty of Arts and Sciences. RISE Research Institutes of Sweden, Linköping.ORCID iD: 0000-0001-7111-2964
2025 (English)Doctoral thesis, comprehensive summary (Other academic)Alternative title
Utformning av system för människa-svärminteraktion (Swedish)
Abstract [en]

Swarms of Unmanned Aerial Vehicles (UAVs, or drones) are envisioned to transform various fields, from emergency response to law enforcement and military operations. Drone swarms provide scalable, adaptable, and decentralized solutions for dynamic work environments. However, the successful integration of these multi-agent systems into real-world settings presents significant challenges, particularly in terms of how humans can safely and effectively interact with and control these systems. Human-Swarm Interaction (HSI) aims to address these challenges by exploring how human operators can manage multiple drones in a cohesive manner, even under highly complex, uncertain conditions.

This thesis studies the problem of designing effective interaction mechanisms and interfaces for human operators to command drone swarms, specifically addressing challenges such as managing a large number of drones, supporting operators’ situational awareness, and balancing between centralized and decentralized control. The research highlights the necessity of rethinking conventional approaches by introducing alternative conceptual models, such as the "choir" metaphor, which re-imagines drone swarms as coordinated, semi-centralized ensembles rather than purely emergent, decentralized collectives. This metaphor aims to balance the collective, often unpredictable behavior of drone swarms with the predictable, directed actions needed in operational environments. By demonstrating how this metaphor can be operationalized in an HSI system architecture, the thesis provides new avenues for conceptualizing human interaction with autonomous systems.

Using a design research approach incorporating multiple-case study and scenario-based design activities to envision future swarm application in dialogue with prospective end users, the thesis develops and evaluates prototypes that embody these nuanced HSI concepts. The interface prototypes draw design inspiration from Real-Time Strategy (RTS) games. These elements include group commands, high-level mission planning, and resource pooling to create a hybrid interaction model that allows operators to maintain both a broad overview and precise control of multiple autonomous and collaborating drones. Domain expert evaluations of these prototypes in contexts such as firefighting and airport management validate the practical utility of these concepts.

The findings emphasize the value of adopting a Human-Technology-Organization (HTO) perspective in the design of HSI systems. Rather than focusing solely on the interaction between humans and technology, this systems-thinking approach acknowledges that drone swarms must be integrated into larger organizational frameworks, such as emergency response command structures or airport ground operations teams. It demonstrates that successful deployment requires accounting for the broader organizational context, including roles, workflows, and coordination needs. This holistic approach to HSI system design ensures that drone swarms not only meet technical performance criteria, such as reliability, responsiveness, and scalability, but also align with human and organizational needs, facilitating their adoption and effective use in a wide range of real-world scenarios. Ultimately, these contributions are intended to bridge the gap between theoretical models of swarm control and practical deployment, advancing both the field of HSI and the broader adoption of drone swarm technologies.

Abstract [sv]

Svärmar av obemannade luftfarkoster (UAV, eller drönare) förväntas omvandla flera områden, exempelvis räddningsinsatser, brottsbekämpning, och militäroperationer. Drönarsvärmar innebar skalbara, anpassningsbara, och decentraliserade lösningar for dynamiska arbetsuppgifter. Den lyckade integreringen av dessa multi-agent-system i verkliga miljöer innebar dock betydande utmaningar, särskilt med avseende på hur människor säkert och effektivt interagerar med och kontrollerar dessa system. Forskningsfältet Människa-Svärm Interaktion (MSI) syftar till att möta dessa utmaningar genom att undersöka hur mänskliga operatorer kan hantera flera drönare på ett sammanhängande vis, även under komplexa och osäkra förhållanden.

Denna avhandling utreder problemet att utforma effektiva och säkra interaktionsmekanismer och gränssnitt for mänskliga operatorer att leda drönarsvärmar, specifikt genom att adressera utmaningar som att hantera ett stort antal drönare, stödja operatorers situationsmedvetenhet, och balansera mellan centraliserad och decentraliserad kontroll. Avhandlingen betonar vikten av att ifrågasatta konventionella tillvägagångssätt genom att introducera alternativa konceptuella modeller, såsom "kör"-metaforen, som omtolkar drönarsvärmar som koordinerade, semicentraliserade ensembler snarare än rent decentraliserade kollektiv. Denna metafor syftar till att balansera det kollektiva, ofta oförutsägbara beteendet hos drönarsvärmar med de förutsägbara, riktade handlingar som behövs i operativa miljöer. Genom att visa hur denna metafor kan operationaliseras i en MSI-systemarkitektur, erbjuder avhandlingen nya sätt att konceptualisera mänsklig interaktion med autonoma system.

Genom att tillämpa en designforskningsmetod som innefattar fallstudier och scenariobaserade designaktiviteter för att föreställa sig framtida svärmtillämpningar i dialog med potentiella slutanvändare, utvecklar och utvärderar avhandlingen prototyper som manifesterar dessa nyanserade MSI-koncept. Gränssnittens prototyper drar designinspiration från realtidsstrategispel (RTS). Dessa element inkluderar enhetshantering och kommandon på gruppnivå, strategisk uppdragsplanering, och resursdelning för att skapa en hybrid interaktionsmodell som gör det möjligt för operatörer att både bibehålla en bred lägesbild och utöva precis kontroll över flera autonoma och samverkande drönare. Domänexperters utvärderingar av dessa prototyper i arbetskontexter som brandbekämpning och flygplatsledning påvisar den praktiska användbarheten av dessa koncept.

Resultaten betonar värdet av att anta ett Människa-Teknik-Organisation (MTO)-perspektiv vid utformningen av MSI-system. Snarare än att enbart fokusera på interaktionen mellan människor och teknik, erkänner detta systemtänkande tillvägagångssätt att drönarsvärmar måste integreras i större organisatoriska ramar, såsom ledningsstrukturer for räddningsinsatser eller markoperativa team på flygplatser. Det visar att framgångsrik implementering av drönarsvärmar kräver att systemutvecklare tar hänsyn till det bredare organisatoriska sammanhanget, inklusive roller, arbetsflöden, och samverkansbehov. Detta holistiska tillvägagångssatt för utformningen av MSI-system säkerställer att drönarsvärmar inte bara uppfyller tekniska prestandakriterier, såsom tillförlitlighet, responsivitet, och skalbarhet, utan också överensstämmer med mänskliga och organisatoriska behov, vilket underlättar deras införande och effektiv användning i en mängd olika tillämpningsscenarier. Över lag är dessa forskningsbidrag avsedda att överbrygga gapet mellan teoretiska modeller för svärmstyrning och praktisk implementering, och därmed avancera och främja både MSI-området och den bredare användningen av svärmteknologier.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2025. , p. 96
Series
Linköping Studies in Arts and Sciences, ISSN 0282-9800 ; 900
Keywords [en]
Human-swarm interaction, Drone swarms, Cognitive systems, Interaction design
Keywords [sv]
Människa-svärminteraktion, Drönarsvärmar, Kognitiva system, Interaktionsdesign
National Category
Human Computer Interaction
Identifiers
URN: urn:nbn:se:liu:diva-211761DOI: 10.3384/9789180759595ISBN: 9789180759588 (print)ISBN: 9789180759595 (electronic)OAI: oai:DiVA.org:liu-211761DiVA, id: diva2:1938952
Public defence
2025-03-27, Ada Lovelace, B Building, Campus Valla, Linköping, 09:15 (English)
Opponent
Supervisors
Available from: 2025-02-20 Created: 2025-02-20 Last updated: 2025-02-20Bibliographically approved
List of papers
1. Drone Swarms in Forest Firefighting: A Local Development Case Study of Multi-Level Human-Swarm Interaction
Open this publication in new window or tab >>Drone Swarms in Forest Firefighting: A Local Development Case Study of Multi-Level Human-Swarm Interaction
Show others...
2020 (English)In: Proceedings of the 11th Nordic Conference on Human-Computer Interaction: Shaping Experiences, Shaping Society, New York, NY, USA: Association for Computing Machinery (ACM), 2020, article id 93Conference paper, Published paper (Refereed)
Abstract [en]

Swarms of autonomous and coordinating Unmanned Aerial Vehicles (UAVs) are rapidly being developed to enable simultaneous control of multiple UAVs. In the field of Human-Swarm Interaction (HSI), researchers develop and study swarm algorithms and various means of control and evaluate their cognitive and task performance. There is, however, a lack of research describing how UAV swarms will fit into future real-world domain contexts. To remedy this, this paper describes a case study conducted within the community of firefighters, more precisely two Swedish fire departments that regularly deploy UAVs in fire responses. Based on an initial description of how their UAVs are used in a forest firefighting context, participating UAV operators and unit commanders envisioned a scenario that showed how the swarm and its capabilities could be utilized given the constraints and requirements of a forest firefighting mission. Based on this swarm scenario description we developed a swarm interaction model that describes how the operators’ interaction traverses multiple levels ranging from the entire swarm, via subswarms and individual UAVs, to specific sensors and equipment carried by the UAVs. The results suggest that human-in-the-loop simulation studies need to enable interaction across multiple swarm levels as this interaction may exert additional cognitive strain on the human operator.

Place, publisher, year, edition, pages
New York, NY, USA: Association for Computing Machinery (ACM), 2020
Keywords
Human-Swarm Interaction, UAV swarm, Firefighting drones, Swarm applications
National Category
Human Computer Interaction Computer graphics and computer vision
Identifiers
urn:nbn:se:liu:diva-177633 (URN)10.1145/3419249.3421239 (DOI)001334839200130 ()2-s2.0-85123040721 (Scopus ID)9781450375795 (ISBN)
Conference
NordiCHI '20
Funder
Swedish Foundation for Strategic Research, FID17-0030
Available from: 2021-06-30 Created: 2021-06-30 Last updated: 2025-10-10Bibliographically approved
2. Swarms, teams, or choirs?: Metaphors in multi-UAV systems design
Open this publication in new window or tab >>Swarms, teams, or choirs?: Metaphors in multi-UAV systems design
2021 (English)In: Advances in Human Factors in Robots, Unmanned Systems and Cybersecurity / [ed] Matteo Zallio, Carlos Raymundo Ibañez, Jesus Hechavarria Hernandez, Cham, 2021, p. 10-15Conference paper, Published paper (Refereed)
Abstract [en]

Future Unmanned Aerial Vehicles (UAVs) are projected to fly and operate in swarms. The swarm metaphor makes explicit and implicit mappings regarding system architecture and human interaction to aspects of natural systems, such as bee societies. Compared to the metaphor of a team, swarming agents as individuals are less capable, more expendable, and more limited in terms of communication and coordination. Given their different features and limitations, the two metaphors could be useful in different scenarios. We also discuss a choir metaphor and illustrate how it can give rise to different design concepts. We conclude that designers and engineers should be mindful of the metaphors they use because they influence—and limit—how to think about and design for multi-UAV systems.

Place, publisher, year, edition, pages
Cham: , 2021
Series
Lecture Notes in Networks and Systems, ISSN 2367-3370 ; 268
Keywords
Drone swarm, Human-swarm interaction, Metaphor
National Category
Human Computer Interaction Computer graphics and computer vision
Identifiers
urn:nbn:se:liu:diva-178162 (URN)10.1007/978-3-030-79997-7_2 (DOI)978-3-030-79996-0 (ISBN)978-3-030-79997-7 (ISBN)
Conference
AHFE 2021, USA, July 25-29, 2021
Funder
Swedish Foundation for Strategic Research, FID17-0030
Available from: 2021-08-09 Created: 2021-08-09 Last updated: 2025-02-20Bibliographically approved
3. Human-Swarm Interaction in Semi-voluntary Search and Rescue Operations: Opportunities and Challenges
Open this publication in new window or tab >>Human-Swarm Interaction in Semi-voluntary Search and Rescue Operations: Opportunities and Challenges
2023 (English)In: Proceedings of the European Conference on Cognitive Ergonomics 2023, New York: Association for Computing Machinery , 2023Conference paper, Published paper (Refereed)
Abstract [en]

In search and rescue (SAR) operations, drones can provide clear and timely situational overview data and object identification. However, the current one-to-one relationship between operators and drones limits scalability. Swarm solutions have been proposed to overcome this limitation, but there are few examples of control concepts for SAR operations. Human-swarm interaction (HSI) presents new challenges in terms of task design, control loops, communication, and managing uncertainty. We present an exploratory study of integrating drone swarms into SAR organizations, with a focus on challenges and opportunities for HSI. Our findings highlight the need for a holistic approach to drone swarm systems design, development, and integration. Careful system and task design is vital to reduce operator workload, maximize situational awareness, and maintaining effective communication among SAR team members. Building trust through technology exposure and training is also important. We identify several key research avenues, including adaptive and intelligent swarm control mechanisms, trust dynamics between operators and swarms, participatory design work, legal and operational frameworks, and the organizational impact of drone swarm integration. Overall, this paper contributes to HSI and SAR research by addressing research gaps concerning the integration effects and constraints of drone swarms in current work settings. The study highlights the potential for implementing drone swarms in semi-voluntary SAR organizations, while emphasizing the importance of considering the tasks and interactions between humans and drones when assessing overall system performance.

Place, publisher, year, edition, pages
New York: Association for Computing Machinery, 2023
Keywords
Search and rescue, Human-swarm interaction, Drone swarm, Cognitive work analysis
National Category
Human Computer Interaction
Identifiers
urn:nbn:se:liu:diva-211760 (URN)10.1145/3605655.3605684 (DOI)9798400708756 (ISBN)
Conference
ECCE 2023: European Conference in Cognitive Ergonomics, Swansea, United Kingdom, September 19 - 22, 2023
Note

Funding agency: The Swedish Foundation for Strategic Research (FID17-0030)

Available from: 2025-02-20 Created: 2025-02-20 Last updated: 2025-02-20Bibliographically approved

Open Access in DiVA

fulltext(15292 kB)1535 downloads
File information
File name FULLTEXT01.pdfFile size 15292 kBChecksum SHA-512
4d36248da37ef0d5473f26f223ea6bc65d6b2cac4344b17931acbd378039bf258551e96f65b689fd550a4cc58a19eab15c4c8e0b4722a9d5ca95d9a32d913eac
Type fulltextMimetype application/pdf
Order online >>

Other links

Publisher's full text

Authority records

Bjurling, Oscar

Search in DiVA

By author/editor
Bjurling, Oscar
By organisation
Human-Centered SystemsFaculty of Arts and Sciences
Human Computer Interaction

Search outside of DiVA

GoogleGoogle Scholar
Total: 1538 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 4319 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf