liu.seSearch for publications in DiVA
System disruptions
We are currently experiencing disruptions on the search portals due to high traffic. We are working to resolve the issue, you may temporarily encounter an error message.
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Light Field Video Compression and Real Time Rendering
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. (Visual computing laboratory)
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering. (Visual computing laboratory)
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.
Show others and affiliations
2019 (English)In: Computer graphics forum (Print), ISSN 0167-7055, E-ISSN 1467-8659, Vol. 38, p. 265-276Article in journal (Refereed) Published
Abstract [en]

Light field imaging is rapidly becoming an established method for generating flexible image based description of scene appearances. Compared to classical 2D imaging techniques, the angular information included in light fields enables effects such as post‐capture refocusing and the exploration of the scene from different vantage points. In this paper, we describe a novel GPU pipeline for compression and real‐time rendering of light field videos with full parallax. To achieve this, we employ a dictionary learning approach and train an ensemble of dictionaries capable of efficiently representing light field video data using highly sparse coefficient sets. A novel, key element in our representation is that we simultaneously compress both image data (pixel colors) and the auxiliary information (depth, disparity, or optical flow) required for view interpolation. During playback, the coefficients are streamed to the GPU where the light field and the auxiliary information are reconstructed using the dictionary ensemble and view interpolation is performed. In order to realize the pipeline we present several technical contributions including a denoising scheme enhancing the sparsity in the dataset which enables higher compression ratios, and a novel pruning strategy which reduces the size of the dictionary ensemble and leads to significant reductions in computational complexity during the encoding of a light field. Our approach is independent of the light field parameterization and can be used with data from any light field video capture system. To demonstrate the usefulness of our pipeline, we utilize various publicly available light field video datasets and discuss the medical application of documenting heart surgery.

Place, publisher, year, edition, pages
John Wiley & Sons, 2019. Vol. 38, p. 265-276
Keywords [en]
Computational photography, Light Fields, Light Fields Compression, Light Field Video
National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:liu:diva-162100DOI: 10.1111/cgf.13835ISI: 000496351100025OAI: oai:DiVA.org:liu-162100DiVA, id: diva2:1371191
Conference
Pacific Graphics 2019
Note

Funding agencies:  childrens heart clinic at Skane University hospital, Barnhjartcentrum; strategic research environment ELLIIT; Swedish Science Council [201505180]; VinnovaVinnova [2017-03728]; Visual Sweden Platform for Augmented Intelligence

Available from: 2019-11-19 Created: 2019-11-19 Last updated: 2021-09-30
In thesis
1. Computational Photography: High Dynamic Range and Light Fields
Open this publication in new window or tab >>Computational Photography: High Dynamic Range and Light Fields
2020 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The introduction and recent advancements of computational photography have revolutionized the imaging industry. Computational photography is a combination of imaging techniques at the intersection of various fields such as optics, computer vision, and computer graphics. These methods enhance the capabilities of traditional digital photography by applying computational techniques both during and after the capturing process. This thesis targets two major subjects in this field: High Dynamic Range (HDR) image reconstruction and Light Field (LF) compressive capturing, compression, and real-time rendering.

The first part of the thesis focuses on the HDR images that concurrently contain detailed information from the very dark shadows to the brightest areas in the scenes. One of the main contributions presented in this thesis is the development of a unified reconstruction algorithm for spatially variant exposures in a single image. This method is based on a camera noise model, and it simultaneously resamples, reconstructs, denoises, and demosaics the image while extending its dynamic range. Furthermore, the HDR reconstruction algorithm is extended to adapt to the local features of the image, as well as the noise statistics, to preserve the high-frequency edges during reconstruction.

In the second part of this thesis, the research focus shifts to the acquisition, encoding, reconstruction, and rendering of light field images and videos in a real-time setting. Unlike traditional integral photography, a light field captures the information of the dynamic environment from all angles, all points in space, and all spectral wavelength and time. This thesis employs sparse representation to provide an end-to-end solution to the problem of encoding, real-time reconstruction, and rendering of high dimensional light field video data sets. These solutions are applied on various types of data sets, such as light fields captured with multi-camera systems or hand-held cameras equipped with micro-lens arrays, and spherical light fields. Finally, sparse representation of light fields was utilized for developing a single sensor light field video camera equipped with a color-coded mask. A new compressive sensing model is presented that is suitable for dynamic scenes with temporal coherency and is capable of reconstructing high-resolution light field videos.  

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2020. p. 122
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 2046
National Category
Other Engineering and Technologies
Identifiers
urn:nbn:se:liu:diva-163693 (URN)10.3384/diss.diva-163693 (DOI)9789179299057 (ISBN)
Public defence
2020-02-28, Domteatern, Visualiseringscenter C, Kungsgatan 54, 60233, Norrköping, 09:15 (English)
Opponent
Supervisors
Available from: 2020-02-18 Created: 2020-02-18 Last updated: 2025-02-18Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Authority records

Hajisharif, SaghiMiandji, EhsanPer, LarssonUnger, Jonas

Search in DiVA

By author/editor
Hajisharif, SaghiMiandji, EhsanPer, LarssonUnger, Jonas
By organisation
Media and Information TechnologyFaculty of Science & Engineering
In the same journal
Computer graphics forum (Print)
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 338 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf