liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Single-frame Regularization for Temporally Stable CNNs
Linköping University, Department of Science and Technology, Media and Information Technology.ORCID iD: 0000-0002-9217-9997
University of Cambridge, England.
Linköping University, Department of Science and Technology, Media and Information Technology.ORCID iD: 0000-0002-7765-1747
2019 (English)In: IEEE Conference on Computer Vision and Pattern Recognition, 2019, p. 11176-11185Conference paper, Published paper (Refereed)
Abstract [en]

Convolutional neural networks (CNNs) can model complicated non-linear relations between images. However, they are notoriously sensitive to small changes in the input. Most CNNs trained to describe image-to-image mappings generate temporally unstable results when applied to video sequences, leading to flickering artifacts and other inconsistencies over time. In order to use CNNs for video material, previous methods have relied on estimating dense frame-to-frame motion information (optical flow) in the training and/or the inference phase, or by exploring recurrent learning structures. We take a different approach to the problem, posing temporal stability as a regularization of the cost function. The regularization is formulated to account for different types of motion that can occur between frames, so that temporally stable CNNs can be trained without the need for video material or expensive motion estimation. The training can be performed as a fine-tuning operation, without architectural modifications of the CNN. Our evaluation shows that the training strategy leads to large improvements in temporal smoothness. Moreover, for small datasets the regularization can help in boosting the generalization performance to a much larger extent than what is possible with naive augmentation strategies.

Place, publisher, year, edition, pages
2019. p. 11176-11185
Keywords [en]
computer vision, machine learning, deep learning, neural networks, image processing
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:liu:diva-162158OAI: oai:DiVA.org:liu-162158DiVA, id: diva2:1371827
Conference
IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, US, June 16-20, 2019
Available from: 2019-11-21 Created: 2019-11-21 Last updated: 2019-11-21

Open Access in DiVA

No full text in DiVA

Other links

CVPR 2019 open accessProject web

Authority records BETA

Unger, Jonas

Search in DiVA

By author/editor
Eilertsen, GabrielUnger, Jonas
By organisation
Media and Information Technology
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 12 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf