liu.seSearch for publications in DiVA
Change search
Link to record
Permanent link

Direct link
BETA
Publications (10 of 28) Show all publications
Unger, J., Hajisharif, S. & Kronander, J. (2016). Unified reconstruction of RAW HDR video data (1sted.). In: Frédéric Dufaux, Patrick Le Callet, Rafal K. Mantiuk, Marta Mrak (Ed.), High dynamic range video: from acquisition to display and applications (pp. 63-82). London, United Kingdom: Academic Press
Open this publication in new window or tab >>Unified reconstruction of RAW HDR video data
2016 (English)In: High dynamic range video: from acquisition to display and applications / [ed] Frédéric Dufaux, Patrick Le Callet, Rafal K. Mantiuk, Marta Mrak, London, United Kingdom: Academic Press, 2016, 1st, p. 63-82Chapter in book (Other academic)
Abstract [en]

Traditional HDR capture has mostly relied on merging images captured with different exposure times. While this works well for static scenes, dynamic scenes poses difficult challenges as registration of differently exposed images often leads to ghosting and other artifacts. This chapter reviews methods which capture HDR-video frames within a single exposure time, using either multiple synchronised sensors, or by multiplexing of the sensor response spatially across the sensor. Most previous HDR reconstruction methods perform demoisaicing, noise reduction, resampling (registration), and HDR-fusion in separate steps. This chapter presents a framework for unified HDR-reconstruction, including all steps in the traditional imaging pipeline in a single adaptive filtering operation, and describes an image formation model and a sensor noise model applicable to both single-, and multi-sensor systems. The benefits of using raw data directly are demonstrated with examples using input data from multiple synchronized sensors, and single images with varying per-pixel gain.

Place, publisher, year, edition, pages
London, United Kingdom: Academic Press, 2016 Edition: 1st
Keywords
High dynamic range imaging, image reconstruction
National Category
Media and Communication Technology
Identifiers
urn:nbn:se:liu:diva-127344 (URN)10.1016/B978-0-08-100412-8.00002-4 (DOI)9780081004128 (ISBN)
Projects
VPS
Funder
Swedish Foundation for Strategic Research , IIS11-0081
Available from: 2016-04-21 Created: 2016-04-21 Last updated: 2018-07-19Bibliographically approved
Hajisharif, S., Kronander, J. & Unger, J. (2015). Adaptive dualISO HDR-reconstruction. EURASIP Journal on Image and Video Processing
Open this publication in new window or tab >>Adaptive dualISO HDR-reconstruction
2015 (English)In: EURASIP Journal on Image and Video Processing, ISSN 1687-5176, E-ISSN 1687-5281Article in journal (Refereed) Published
Abstract [en]

With the development of modern image sensors enabling flexible image acquisition, single shot HDR imaging is becoming increasingly popular. In this work we capture single shot HDR images using an imaging sensor with spatially varying gain/ISO. In comparison to previous single shot HDR capture based on a single sensor, this allows all incoming photons to be used in the imaging, instead of wasting incoming light using spatially varying ND-filters, commonly used in previous works. The main technical contribution in this work is an  extension of previous HDR reconstruction approaches for single shot HDR imaging based on local polynomial approximations [15,10]. Using a sensor noise model, these works deploy a statistically informed filtering operation to reconstruct HDR pixel values. However, instead of using a fixed filter size, we introduce two novel algorithms for adaptive filter kernel selection. Unlike previous works, using  adaptive filter kernels [16], our algorithms are based on analysing the model fit and the expected statistical deviation of the estimate based on the sensor noise model. Using an iterative procedure we can then adapt the filter kernel according to the image structure and the statistical image noise. Experimental results show that the proposed filter de-noises the noisy image carefully while well preserving the important image features such as edges and corners, outperforming previous methods. To demonstrate the robustness of our approach, we have exploited input images from raw sensor data using a commercial off-the shelf camera. To further analyze our algorithm, we have also implemented a camera simulator to evaluate different gain pattern and noise properties of the sensor.

Place, publisher, year, edition, pages
Springer Publishing Company, 2015
Keywords
HDR reconstruction; Single shot HDR imaging; DualISO; Statistical image fitlering
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:liu:diva-122587 (URN)10.1186/s13640-015-0095-0 (DOI)000366324500001 ()
Note

Funding agencies: Swedish Foundation for Strategic Research (SSF) [IIS11-0081]; Linkoping University Center for Industrial Information Technology (CENIIT); Swedish Research Council through the Linnaeus Environment CADICS

Available from: 2015-11-10 Created: 2015-11-10 Last updated: 2018-01-10Bibliographically approved
Miandji, E., Kronander, J. & Unger, J. (2015). Compressive Image Reconstruction in Reduced Union of Subspaces. Paper presented at Eurographics 2015. Computer Graphics Forum, 34(2), 33-44
Open this publication in new window or tab >>Compressive Image Reconstruction in Reduced Union of Subspaces
2015 (English)In: Computer Graphics Forum, ISSN 1467-8659, Vol. 34, no 2, p. 33-44Article in journal (Refereed) Published
Abstract [en]

We present a new compressed sensing framework for reconstruction of incomplete and possibly noisy images and their higher dimensional variants, e.g. animations and light-fields. The algorithm relies on a learning-based basis representation. We train an ensemble of intrinsically two-dimensional (2D) dictionaries that operate locally on a set of 2D patches extracted from the input data. We show that one can convert the problem of 2D sparse signal recovery to an equivalent 1D form, enabling us to utilize a large family of sparse solvers. The proposed framework represents the input signals in a reduced union of subspaces model, while allowing sparsity in each subspace. Such a model leads to a much more sparse representation than widely used methods such as K-SVD. To evaluate our method, we apply it to three different scenarios where the signal dimensionality varies from 2D (images) to 3D (animations) and 4D (light-fields). We show that our method outperforms state-of-the-art algorithms in computer graphics and image processing literature.

Place, publisher, year, edition, pages
John Wiley & Sons Ltd, 2015
Keywords
Image reconstruction, compressed sensing, light field imaging
National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-119639 (URN)10.1111/cgf.12539 (DOI)000358326600008 ()
Conference
Eurographics 2015
Projects
VPS
Funder
Swedish Foundation for Strategic Research , IIS11-0081
Available from: 2015-06-23 Created: 2015-06-23 Last updated: 2018-11-23Bibliographically approved
Kronander, J., Banterle, F., Gardner, A., Miandji, E. & Unger, J. (2015). Photorealistic rendering of mixed reality scenes. Paper presented at The 36th Annual Conference of the European Association of Computer Graphics, Eurographics 2015, Zürich, Switzerland, 4th–8th May 2015. Computer graphics forum (Print), 34(2), 643-665
Open this publication in new window or tab >>Photorealistic rendering of mixed reality scenes
Show others...
2015 (English)In: Computer graphics forum (Print), ISSN 0167-7055, E-ISSN 1467-8659, Vol. 34, no 2, p. 643-665Article in journal (Refereed) Published
Abstract [en]

Photo-realistic rendering of virtual objects into real scenes is one of the most important research problems in computer graphics. Methods for capture and rendering of mixed reality scenes are driven by a large number of applications, ranging from augmented reality to visual effects and product visualization. Recent developments in computer graphics, computer vision, and imaging technology have enabled a wide range of new mixed reality techniques including methods of advanced image based lighting, capturing spatially varying lighting conditions, and algorithms for seamlessly rendering virtual objects directly into photographs without explicit measurements of the scene lighting. This report gives an overview of the state-of-the-art in this field, and presents a categorization and comparison of current methods. Our in-depth survey provides a tool for understanding the advantages and disadvantages of each method, and gives an overview of which technique is best suited to a specific problem.

Place, publisher, year, edition, pages
Wiley-Blackwell, 2015
Keywords
Picture/Image Generation—Illumination Estimation, Image-Based Lighting, Reflectance and Shading
National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-118542 (URN)10.1111/cgf.12591 (DOI)000358326600060 ()
Conference
The 36th Annual Conference of the European Association of Computer Graphics, Eurographics 2015, Zürich, Switzerland, 4th–8th May 2015
Projects
VPS
Funder
Swedish Foundation for Strategic Research , IIS11-0081Linnaeus research environment CADICS
Available from: 2015-05-31 Created: 2015-05-31 Last updated: 2017-12-04Bibliographically approved
Kronander, J. (2015). Physically Based Rendering of Synthetic Objects in Real Environments. (Doctoral dissertation). Linköping: Linköping University Electronic Press
Open this publication in new window or tab >>Physically Based Rendering of Synthetic Objects in Real Environments
2015 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis presents methods for photorealistic rendering of virtual objects so that they can be seamlessly composited into images of the real world. To generate predictable and consistent results, we study physically based methods, which simulate how light propagates in a mathematical model of the augmented scene. This computationally challenging problem demands both efficient and accurate simulation of the light transport in the scene, as well as detailed modeling of the geometries, illumination conditions, and material properties. In this thesis, we discuss and formulate the challenges inherent in these steps and present several methods to make the process more efficient.

In particular, the material contained in this thesis addresses four closely related areas: HDR imaging, IBL, reflectance modeling, and efficient rendering. The thesis presents a new, statistically motivated algorithm for HDR reconstruction from raw camera data combining demosaicing, denoising, and HDR fusion in a single processing operation. The thesis also presents practical and robust methods for rendering with spatially and temporally varying illumination conditions captured using omnidirectional HDR video. Furthermore, two new parametric BRDF models are proposed for surfaces exhibiting wide angle gloss. Finally, the thesis also presents a physically based light transport algorithm based on Markov Chain Monte Carlo methods that allows approximations to be used in place of exact quantities, while still converging to the exact result. As illustrated in the thesis, the proposed algorithm enables efficient rendering of scenes with glossy transfer and heterogenous participating media.

Abstract [sv]

En av de största utmaningarna inom datorgrafik är att syntetisera, eller rendera, fotorealistiska bilder. Fotorealistisk rendering används idag inom många tillämpningsområden såsom specialeffekter i film, datorspel, produktvisualisering och virtuell verklighet. I många praktiska tillämpningar av fotorealistisk rendering är det viktigt att kunna placera in virtuella objekt i fotografier, så att de virtuella objekten ser verkliga ut. IKEA-katalogen, till exempel, produceras i många olika versioner för att passa olika länder och regioner. Grunden till de flesta bilderna i katalogen är oftast densamma, men symboler och standardmått på möbler varierar ofta för olika versioner av katalogen. Istället för att fotografera varje version separat kan man använda ett grundfotografi och lägga in olika virtuella objekt såsom möbler i fotot. Genom att på det här sättet möblera ett rum virtuellt, istället för på riktigt, kan man också snabbt testa olika möbleringar och därmed göra ekonomiska besparingar.

Den här avhandlingen bidrar med metoder och algoritmer för att rendera fotorealistiska bilder av virtuella objekt som kan blandas med verkliga fotografier. För att rendera sådana bilder används fysikaliskt baserade simuleringar av hur ljus interagerar med virtuella och verkliga objekt i motivet. För fotorealistiska resultat kräver simuleringarna noggrann modellering av objektens geometri, belysning och materialegenskaper, såsom färg, textur och reflektans.

För att de virtuella objekten ska se verkliga ut är det viktigt att belysa dem med samma ljus som de skulle ha haft om de var en del av den verkliga miljön. Därför är det viktigt att noggrant mäta och modellera ljusförhållanden på de platser i scenen där de virtuella objekten ska placeras. För detta använder vi High Dynamic Range-fotografi, eller HDR. Med hjälp av HDR-fotografi kan vi noggrant mäta hela omfånget av det infallande ljuset i en punkt, från mörka skuggor till direkta ljuskällor. Detta är inte möjligt med traditionella digitalkameror, då det dynamiska omfånget hos vanliga kamerasensorer är begränsat. Avhandlingen beskriver nya metoder för att rekonstruera HDR-bilder som ger mindre brus och artefakter än tidigare metoder. Vi presenterar också metoder för att rendera virtuella objekt som rör sig mellan regioner med olika belysning, eller där belysningen varierar i tiden. Metoder för att representera spatiellt varierande belysning på ett kompakt sätt presenteras också. För att noggrant beskriva hur glansiga ytor sprider eller reflekterar ljus, beskrivs också två nya parametriska modeller som är mer verklighetstrogna än tidigare reflektionsmodeller. I avhandlingen presenteras också en ny metod för effektiv rendering av motiv som är mycket beräkningskrävande, till exempel scener med uppmätta belysningsförhållanden, komplicerade  material, och volumetriska modeller som rök, moln, textiler, biologisk vävnad och vätskor. Metoden bygger på en typ av så kallade Markov Chain Monte Carlo metoder för att simulera ljustransporten i scenen, och är inspirerad av nyligen presenterade resultat inom matematisk statistik.

Metoderna som beskrivs i avhandlingen presenteras i kontexten av fotorealistisk rendering av virtuella objekt i riktiga miljöer, då majoriteten av forskningen utförts inom detta område. Flera av de metoder som presenteras i denna avhandling är dock tillämpbara inom andra domäner, såsom fysiksimulering, datorseende och vetenskaplig visualisering.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2015. p. 135
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 1717
National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-122588 (URN)10.3384/diss.diva-122588 (DOI)978-91-7685-912-4 (ISBN)
Public defence
2015-12-04, Domteatern, Visualiseringscenter C, Kungsgatan 54, Norrköping, 09:15 (English)
Opponent
Supervisors
Available from: 2015-11-10 Created: 2015-11-10 Last updated: 2019-11-15Bibliographically approved
Kronander, J., Schön, T. B. & Unger, J. (2015). Pseudo-Marginal Metropolis Light Transport. In: Proceeding SA '15 SIGGRAPH Asia 2015 Technical Briefs: . Paper presented at The 8th ACM SIGGRAPH Conference and Exhibition, Asia Technical Briefs, 3-5 November, Kobe, Japan (pp. 13:1-13:4). ACM Digital Library
Open this publication in new window or tab >>Pseudo-Marginal Metropolis Light Transport
2015 (English)In: Proceeding SA '15 SIGGRAPH Asia 2015 Technical Briefs, ACM Digital Library, 2015, p. 13:1-13:4Conference paper, Published paper (Other academic)
Abstract [en]

Accurate and efficient simulation of light transport in heterogeneous participating media, such as smoke, clouds and fire, plays a key role in the synthesis of visually interesting renderings for e.g. visual effects, computer games and product visualization. However, rendering of scenes with heterogenous participating with Metropolis light transport (MLT) algorithms have previously been limited to primary sample space methods or using biased approximations of the transmittance in the scene. This paper presents a new sampling strategy for Markov chain Monte Carlo (MCMC) methods, e.g. MLT, based on pseudo-marginal MCMC. Specifically, we show that any positive and unbiased estimator of the target distribution can replace the exact quantity to simulate a Markov Chain with a stationary distribution that has a marginal which is the exact target distribution of interest. This enables us to evaluate the transmittance function with recent unbiased estimators which leads to significantly shorter rendering times. Compared to previous work, relying on (biased) ray-marching for evaluating transmittance, our method enables simulation of longer Markov chains, a better exploration of the path space, and consequently less image noise, for a given computational budget. To demonstrate the usefulness of our pseudo-marginal approach, we compare it to representative methods for efficient rendering of anisotropic heterogeneous participating media and glossy transfer. We show that it performs significantly better in terms of image noise and rendering times compared to previous techniques. Our method is robust, and can easily be implemented in a modern renderer.

Place, publisher, year, edition, pages
ACM Digital Library, 2015
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:liu:diva-122586 (URN)10.1145/2820903.2820922 (DOI)978-1-4503-3930-8 (ISBN)
Conference
The 8th ACM SIGGRAPH Conference and Exhibition, Asia Technical Briefs, 3-5 November, Kobe, Japan
Available from: 2015-11-10 Created: 2015-11-10 Last updated: 2018-01-10Bibliographically approved
Kronander, J., Gustavson, S., Bonnet, G., Ynnerman, A. & Unger, J. (2014). A unified framework for multi-sensor HDR video reconstruction. Signal Processing : Image Communications, 29(2), 203-215
Open this publication in new window or tab >>A unified framework for multi-sensor HDR video reconstruction
Show others...
2014 (English)In: Signal Processing : Image Communications, ISSN 0923-5965, Vol. 29, no 2, p. 203-215Article in journal (Refereed) Published
Abstract [en]

One of the most successful approaches to modern high quality HDR-video capture is to use camera setups with multiple sensors imaging the scene through a common optical system. However, such systems pose several challenges for HDR reconstruction algorithms. Previous reconstruction techniques have considered debayering, denoising, resampling (alignment) and exposure fusion as separate problems. In contrast, in this paper we present a unifying approach, performing HDR assembly directly from raw sensor data. Our framework includes a camera noise model adapted to HDR video and an algorithm for spatially adaptive HDR reconstruction based on fitting of local polynomial approximations to observed sensor data. The method is easy to implement and allows reconstruction to an arbitrary resolution and output mapping. We present an implementation in CUDA and show real-time performance for an experimental 4 Mpixel multi-sensor HDR video system. We further show that our algorithm has clear advantages over existing methods, both in terms of flexibility and reconstruction quality.

Place, publisher, year, edition, pages
Elsevier, 2014
Keywords
HDR video, HDR fusion, Kernel regression, Radiometric calibration
National Category
Media Engineering
Identifiers
urn:nbn:se:liu:diva-104617 (URN)10.1016/j.image.2013.08.018 (DOI)000332999200003 ()
Projects
VPS
Funder
Swedish Foundation for Strategic Research , IIS11-0081
Available from: 2014-02-19 Created: 2014-02-19 Last updated: 2015-11-10Bibliographically approved
Kronander, J., Schön, T. B. & Dahlin, J. (2014). Backward sequential Monte Carlo for marginal smoothing. In: Proceedings of the 2014 IEEE Statistical Signal Processing Workshop: . Paper presented at 2014 IEEE Statistical Signal Processing Workshop (SSP), 29 June - 02 July 2014, Gold Coast, Australia (pp. 368-371). IEEE Press
Open this publication in new window or tab >>Backward sequential Monte Carlo for marginal smoothing
2014 (English)In: Proceedings of the 2014 IEEE Statistical Signal Processing Workshop, IEEE Press, 2014, p. 368-371Conference paper, Published paper (Refereed)
Abstract [en]

In this paper we propose a new type of particle smoother with linear computational complexity. The smoother is based on running a sequential Monte Carlo sampler backward in time after an initial forward filtering pass. While this introduces dependencies among the backward trajectories we show through simulation studies that the new smoother can outperform existing forward-backward particle smoothers when targeting the marginal smoothing densities.

Place, publisher, year, edition, pages
IEEE Press, 2014
National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-107138 (URN)10.1109/SSP.2014.6884652 (DOI)000361019700093 ()
Conference
2014 IEEE Statistical Signal Processing Workshop (SSP), 29 June - 02 July 2014, Gold Coast, Australia
Projects
VPS, CADICS
Available from: 2014-06-05 Created: 2014-06-05 Last updated: 2016-05-04
Hajsharif, S., Kronander, J. & Unger, J. (2014). HDR reconstruction for alternating gain (ISO) sensor readout. In: Eurographics 2014 short papers: . Paper presented at Eurographics, Strasbourg, France, April 7-11, 2014.
Open this publication in new window or tab >>HDR reconstruction for alternating gain (ISO) sensor readout
2014 (English)In: Eurographics 2014 short papers, 2014Conference paper, Published paper (Refereed)
Abstract [en]

Modern image sensors are becoming more and more flexible in the way an image is captured. In this paper, we focus on sensors that allow the per pixel gain to be varied over the sensor and develop a new technique for efficient and accurate reconstruction of high dynamic range (HDR) images based on such input data. Our method estimates the radiant power at each output pixel using a sampling operation which performs color interpolation, re-sampling, noise reduction and HDR-reconstruction in a single step. The reconstruction filter uses a sensor noise model to weight the input pixel samples according to their variances. Our algorithm works in only a small spatial neighbourhood around each pixel and lends itself to efficient implementation in hardware. To demonstrate the utility of our approach we show example HDR-images reconstructed from raw sensor data captured using off-the shelf consumer hardware which allows for two different gain settings for different rows in the same image. To analyse the accuracy of the algorithm, we also use synthetic images from a camera simulation software.

Keywords
HDR, image reconstruction, dual-ISO, image processing
National Category
Computer Sciences
Identifiers
urn:nbn:se:liu:diva-104922 (URN)
Conference
Eurographics, Strasbourg, France, April 7-11, 2014
Projects
VPS
Available from: 2014-03-03 Created: 2014-03-03 Last updated: 2018-01-11Bibliographically approved
Kronander, J., Dahlin, J., Jönsson, D., Kok, M., Schön, T. & Unger, J. (2014). Real-time video based lighting using GPU raytracing. In: Proceedings of the 22nd European Signal Processing Conference (EUSIPCO), 2014: . Paper presented at 22nd European Signal Processing Conference (EUSIPCO 2014), 1-5 September 2014, Lisbon, Portugal. IEEE Signal Processing Society
Open this publication in new window or tab >>Real-time video based lighting using GPU raytracing
Show others...
2014 (English)In: Proceedings of the 22nd European Signal Processing Conference (EUSIPCO), 2014, IEEE Signal Processing Society, 2014Conference paper, Published paper (Refereed)
Abstract [en]

The recent introduction of HDR video cameras has enabled the development of image based lighting techniques for rendering virtual objects illuminated with temporally varying real world illumination. A key challenge in this context is that rendering realistic objects illuminated with video environment maps is computationally demanding. In this work, we present a GPU based rendering system based on the NVIDIA OptiX framework, enabling real time raytracing of scenes illuminated with video environment maps. For this purpose, we explore and compare several Monte Carlo sampling approaches, including bidirectional importance sampling, multiple importance sampling and sequential Monte Carlo samplers. While previous work have focused on synthetic data and overly simple environment maps sequences, we have collected a set of real world dynamic environment map sequences using a state-of-art HDR video camera for evaluation and comparisons.

Place, publisher, year, edition, pages
IEEE Signal Processing Society, 2014
Series
Proceedings of the European Signal Processing Conference, ISSN 2076-1465
Keywords
High dynamic range imaging, image synthesis, iamge based lighting
National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-107638 (URN)000393420200327 ()
Conference
22nd European Signal Processing Conference (EUSIPCO 2014), 1-5 September 2014, Lisbon, Portugal
Projects
VPS
Funder
Swedish Foundation for Strategic Research , IISS-0081
Available from: 2014-06-17 Created: 2014-06-17 Last updated: 2018-07-19Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-6071-2507

Search in DiVA

Show all publications