liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Cognitive Learning, Monitoring and Assistance of Industrial Workflows Using Egocentric Sensor Networks
Department Augmented Vision, German Research Center for Artificial Intelligence, Kaiserslautern, Germany; Department of Computer Science, Technical University of Kaiserslautern, Kaiserslautern, Germany.
Department of Computer Science, University of Bristol, Bristol, UK.
School of Computing, University of Leeds, Leeds, UK; Department of Computing, Edge Hill University, Ormskirk, UK.
Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.ORCID iD: 0000-0002-1971-4295
Show others and affiliations
2015 (English)In: PLoS ONE, ISSN 1932-6203, E-ISSN 1932-6203, Vol. 10, no 6, e0127769Article in journal (Refereed) Published
Abstract [en]

Today, the workflows that are involved in industrial assembly and production activities are becoming increasingly complex. To efficiently and safely perform these workflows is demanding on the workers, in particular when it comes to infrequent or repetitive tasks. This burden on the workers can be eased by introducing smart assistance systems. This article presents a scalable concept and an integrated system demonstrator designed for this purpose. The basic idea is to learn workflows from observing multiple expert operators and then transfer the learnt workflow models to novice users. Being entirely learning-based, the proposed system can be applied to various tasks and domains. The above idea has been realized in a prototype, which combines components pushing the state of the art of hardware and software designed with interoperability in mind. The emphasis of this article is on the algorithms developed for the prototype: 1) fusion of inertial and visual sensor information from an on-body sensor network (BSN) to robustly track the user’s pose in magnetically polluted environments; 2) learning-based computer vision algorithms to map the workspace, localize the sensor with respect to the workspace and capture objects, even as they are carried; 3) domain-independent and robust workflow recovery and monitoring algorithms based on spatiotemporal pairwise relations deduced from object and user movement with respect to the scene; and 4) context-sensitive augmented reality (AR) user feedback using a head-mounted display (HMD). A distinguishing key feature of the developed algorithms is that they all operate solely on data from the on-body sensor network and that no external instrumentation is needed. The feasibility of the chosen approach for the complete action-perception-feedback loop is demonstrated on three increasingly complex datasets representing manual industrial tasks. These limited size datasets indicate and highlight the potential of the chosen technology as a combined entity as well as point out limitations of the system.

Place, publisher, year, edition, pages
2015. Vol. 10, no 6, e0127769
Keyword [en]
body pose estimation; computer vision; workflow recognition; augmented reality; computer assisted instruction; human computer interaction
National Category
Signal Processing Control Engineering Human Computer Interaction Information Systems Other Computer and Information Science
Identifiers
URN: urn:nbn:se:liu:diva-119958DOI: 10.1371/journal.pone.0127769ISI: 000358151300003PubMedID: 26126116OAI: oai:DiVA.org:liu-119958DiVA: diva2:838724
Available from: 2015-07-01 Created: 2015-07-01 Last updated: 2017-12-04

Open Access in DiVA

fulltext(7317 kB)340 downloads
File information
File name FULLTEXT01.pdfFile size 7317 kBChecksum SHA-512
094827de225007d2365bde5a8f0a4f441288cd9d16da15c42024fd67d3131fd4d1af6bf96e8c3418dc644f63122dc498bedda947a147cc684bbbaa8f70b58044
Type fulltextMimetype application/pdf

Other links

Publisher's full textPubMedhttp://journals.plos.org/plosone/article?id=10.1371/journal.pone.0127769

Authority records BETA

Hendeby, Gustaf

Search in DiVA

By author/editor
Hendeby, Gustaf
By organisation
Automatic ControlThe Institute of Technology
In the same journal
PLoS ONE
Signal ProcessingControl EngineeringHuman Computer InteractionInformation SystemsOther Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 340 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
pubmed
urn-nbn

Altmetric score

doi
pubmed
urn-nbn
Total: 484 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf