liu.seSearch for publications in DiVA
Change search
ReferencesLink to record
Permanent link

Direct link
Structure and Motion Estimation from Rolling Shutter Video
Linköping University, Department of Electrical Engineering, Computer Vision. Linköping University, The Institute of Technology.
Linköping University, Department of Electrical Engineering, Computer Vision. Linköping University, The Institute of Technology.
Linköping University, Department of Electrical Engineering, Computer Vision. Linköping University, The Institute of Technology.ORCID iD: 0000-0002-5698-5983
Linköping University, Department of Electrical Engineering, Computer Vision. Linköping University, The Institute of Technology.ORCID iD: 0000-0002-6096-3648
2011 (English)In: IEEE International Conference onComputer Vision Workshops (ICCV Workshops), 2011, IEEE Xplore , 2011, 17-23 p.Conference paper (Refereed)
Abstract [en]

The majority of consumer quality cameras sold today have CMOS sensors with rolling shutters. In a rolling shutter camera, images are read out row by row, and thus each row is exposed during a different time interval. A rolling-shutter exposure causes geometric image distortions when either the camera or the scene is moving, and this causes state-of-the-art structure and motion algorithms to fail. We demonstrate a novel method for solving the structure and motion problem for rolling-shutter video. The method relies on exploiting the continuity of the camera motion, both between frames, and across a frame. We demonstrate the effectiveness of our method by controlled experiments on real video sequences. We show, both visually and quantitatively, that our method outperforms standard structure and motion, and is more accurate and efficient than a two-step approach, doing image rectification and structure and motion.

Place, publisher, year, edition, pages
IEEE Xplore , 2011. 17-23 p.
Keyword [en]
Structure and Motion, Rolling Shutter, Bundel Adjustment
National Category
Computer Vision and Robotics (Autonomous Systems)
URN: urn:nbn:se:liu:diva-75258DOI: 10.1109/ICCVW.2011.6130217ISBN: 978-1-4673-0062-9OAI: diva2:505440
2nd IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 2011,6-13 November,Barcelona, Spain
Available from: 2012-03-01 Created: 2012-02-23 Last updated: 2016-05-04Bibliographically approved
In thesis
1. Motion and Structure Estimation From Video
Open this publication in new window or tab >>Motion and Structure Estimation From Video
2012 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Digital camera equipped cell phones were introduced in Japan in 2001, they quickly became popular and by 2003 outsold the entire stand-alone digital camera market. In 2010 sales passed one billion units and the market is still growing. Another trend is the rising popularity of smartphones which has led to a rapid development of the processing power on a phone, and many units sold today bear close resemblance to a personal computer. The combination of a powerful processor and a camera which is easily carried in your pocket, opens up a large eld of interesting computer vision applications.

The core contribution of this thesis is the development of methods that allow an imaging device such as the cell phone camera to estimates its own motion and to capture the observed scene structure. One of the main focuses of this thesis is real-time performance, where a real-time constraint does not only result in shorter processing times, but also allows for user interaction.

In computer vision, structure from motion refers to the process of estimating camera motion and 3D structure by exploring the motion in the image plane caused by the moving camera. This thesis presents several methods for estimating camera motion. Given the assumption that a set of images has known camera poses associated to them, we train a system to solve the camera pose very fast for a new image. For the cases where no a priory information is available a fast minimal case solver is developed. The solver uses ve points in two camera views to estimate the cameras relative position and orientation. This type of minimal case solver is usually used within a RANSAC framework. In order to increase accuracy and performance a renement to the random sampling strategy of RANSAC is proposed. It is shown that the new scheme doubles the performance for the ve point solver used on video data. For larger systems of cameras a new Bundle Adjustment method is developed which are able to handle video from cell phones.

Demands for reduction in size, power consumption and price has led to a redesign of the image sensor. As a consequence the sensors have changed from a global shutter to a rolling shutter, where a rolling shutter image is acquired row by row. Classical structure from motion methods are modeled on the assumption of a global shutter and a rolling shutter can severely degrade their performance. One of the main contributions of this thesis is a new Bundle Adjustment method for cameras with a rolling shutter. The method accurately models the camera motion during image exposure with an interpolation scheme for both position and orientation.

The developed methods are not restricted to cellphones only, but is rather applicable to any type of mobile platform that is equipped with cameras, such as a autonomous car or a robot. The domestic robot comes in many  avors, everything from vacuum cleaners to service and pet robots. A robot equipped with a camera that is capable of estimating its own motion while sensing its environment, like the human eye, can provide an eective means of navigation for the robot. Many of the presented methods are well suited of robots, where low latency and real-time constraints are crucial in order to allow them to interact with their environment.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2012. 42 p.
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 1449
National Category
Engineering and Technology
urn:nbn:se:liu:diva-76904 (URN)978-91-7519-892-7 (ISBN)
Public defence
2012-05-16, Visionen, hus B, Campus Valla, Linköpings Universitet, Linköping, 13:15 (English)
Virtual Photo Set (VPS)
Available from: 2012-04-24 Created: 2012-04-24 Last updated: 2016-05-04Bibliographically approved

Open Access in DiVA

fulltext(2743 kB)1904 downloads
File information
File name FULLTEXT01.pdfFile size 2743 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Search in DiVA

By author/editor
Hedborg, JohanRingaby, ErikForssén, Per-ErikFelsberg, Michael
By organisation
Computer VisionThe Institute of Technology
Computer Vision and Robotics (Autonomous Systems)

Search outside of DiVA

GoogleGoogle Scholar
Total: 1904 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 439 hits
ReferencesLink to record
Permanent link

Direct link