liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Vision-based Localization and Attitude Estimation Methods in Natural Environments
Linköping University, Department of Electrical Engineering, Computer Vision. Linköping University, Faculty of Science & Engineering.
2019 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Over the last decade, the usage of unmanned systems such as Unmanned Aerial Vehicles (UAVs), Unmanned Surface Vessels (USVs) and Unmanned Ground Vehicles (UGVs) has increased drastically, and there is still a rapid growth. Today, unmanned systems are being deployed in many daily operations, e.g. for deliveries in remote areas, to increase efficiency of agriculture, and for environmental monitoring at sea. For safety reasons, unmanned systems are often the preferred choice for surveillance missions in hazardous environments, e.g. for detection of nuclear radiation, and in disaster areas after earthquakes, hurricanes, or during forest fires. For safe navigation of the unmanned systems during their missions, continuous and accurate global localization and attitude estimation is mandatory.

Over the years, many vision-based methods for position estimation have been developed, primarily for urban areas. In contrast, this thesis is mainly focused on vision-based methods for accurate position and attitude estimates in natural environments, i.e. beyond the urban areas. Vision-based methods possess several characteristics that make them appealing as global position and attitude sensors. First, vision sensors can be realized and tailored for most unmanned vehicle applications. Second, geo-referenced terrain models can be generated worldwide from satellite imagery and can be stored onboard the vehicles. In natural environments, where the availability of geo-referenced images in general is low, registration of image information with terrain models is the natural choice for position and attitude estimation. This is the problem area that I addressed in the contributions of this thesis.

The first contribution is a method for full 6DoF (degrees of freedom) pose estimation from aerial images. A dense local height map is computed using structure from motion. The global pose is inferred from the 3D similarity transform between the local height map and a digital elevation model. Aligning height information is assumed to be more robust to season variations than feature-based matching.

The second contribution is a method for accurate attitude (pitch and roll angle) estimation via horizon detection. It is one of only a few methods that use an omnidirectional (fisheye) camera for horizon detection in aerial images. The method is based on edge detection and a probabilistic Hough voting scheme. The method allows prior knowledge of the attitude angles to be exploited to make the initial attitude estimates more robust. The estimates are then refined through registration with the geometrically expected horizon line from a digital elevation model. To the best of our knowledge, it is the first method where the ray refraction in the atmosphere is taken into account, which enables the highly accurate attitude estimates.

The third contribution is a method for position estimation based on horizon detection in an omnidirectional panoramic image around a surface vessel. Two convolutional neural networks (CNNs) are designed and trained to estimate the camera orientation and to segment the horizon line in the image. The MOSSE correlation filter, normally used in visual object tracking, is adapted to horizon line registration with geometric data from a digital elevation model. Comprehensive field trials conducted in the archipelago demonstrate the GPS-level accuracy of the method, and that the method can be trained on images from one region and then applied to images from a previously unvisited test area.

The CNNs in the third contribution apply the typical scheme of convolutions, activations, and pooling. The fourth contribution focuses on the activations and suggests a new formulation to tune and optimize a piecewise linear activation function during training of CNNs. Improved classification results from experiments when tuning the activation function led to the introduction of a new activation function, the Shifted Exponential Linear Unit (ShELU).

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2019. , p. 81
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 1977
National Category
Computer Vision and Robotics (Autonomous Systems)
Identifiers
URN: urn:nbn:se:liu:diva-154159DOI: 10.3384/diss.diva-154159ISBN: 9789176851180 (print)OAI: oai:DiVA.org:liu-154159DiVA, id: diva2:1303454
Public defence
2019-04-26, Domen, Visualiseringscenter C, Campus Norrköping, Norrköping, 13:15 (English)
Opponent
Supervisors
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)Available from: 2019-04-12 Created: 2019-04-09 Last updated: 2019-04-30Bibliographically approved
List of papers
1. Efficient 7D Aerial Pose Estimation
Open this publication in new window or tab >>Efficient 7D Aerial Pose Estimation
2013 (English)In: 2013 IEEE Workshop on Robot Vision (WORV), IEEE , 2013, p. 88-95Conference paper, Published paper (Refereed)
Abstract [en]

A method for online global pose estimation of aerial images by alignment with a georeferenced 3D model is presented.Motion stereo is used to reconstruct a dense local height patch from an image pair. The global pose is inferred from the 3D transform between the local height patch and the model.For efficiency, the sought 3D similarity transform is found by least-squares minimizations of three 2D subproblems.The method does not require any landmarks or reference points in the 3D model, but an approximate initialization of the global pose, in our case provided by onboard navigation sensors, is assumed.Real aerial images from helicopter and aircraft flights are used to evaluate the method. The results show that the accuracy of the position and orientation estimates is significantly improved compared to the initialization and our method is more robust than competing methods on similar datasets.The proposed matching error computed between the transformed patch and the map clearly indicates whether a reliable pose estimate has been obtained.

Place, publisher, year, edition, pages
IEEE, 2013
Keywords
Pose estimation, aerial images, registration, 3D model
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:liu:diva-89477 (URN)10.1109/WORV.2013.6521919 (DOI)000325279400014 ()978-1-4673-5646-6 (ISBN)978-1-4673-5647-3 (ISBN)
Conference
IEEE Workshop on Robot Vision 2013, Clearwater Beach, Florida, USA, January 16-17, 2013
Available from: 2013-02-26 Created: 2013-02-26 Last updated: 2019-04-12
2. Probabilistic Hough Voting for Attitude Estimation from Aerial Fisheye Images
Open this publication in new window or tab >>Probabilistic Hough Voting for Attitude Estimation from Aerial Fisheye Images
2013 (English)In: Image Analysis: 18th Scandinavian Conference, SCIA 2013, Espoo, Finland, June 17-20, 2013. Proceedings / [ed] Joni-Kristian Kämäräinen and Markus Koskela, Springer Berlin/Heidelberg, 2013, p. 478-488Conference paper, Published paper (Refereed)
Abstract [en]

For navigation of unmanned aerial vehicles (UAVs), attitude estimation is essential. We present a method for attitude estimation (pitch and roll angle) from aerial fisheye images through horizon detection. The method is based on edge detection and a probabilistic Hough voting scheme.  In a flight scenario, there is often some prior knowledge of the vehicle altitude and attitude. We exploit this prior to make the attitude estimation more robust by letting the edge pixel votes be weighted based on the probability distributions for the altitude and pitch and roll angles. The method does not require any sky/ground segmentation as most horizon detection methods do. Our method has been evaluated on aerial fisheye images from the internet. The horizon is robustly detected in all tested images. The deviation in the attitude estimate between our automated horizon detection and a manual detection is less than 1 degree.

Place, publisher, year, edition, pages
Springer Berlin/Heidelberg, 2013
Series
Lecture Notes in Computer Science, ISSN 0302-9743, E-ISSN 1611-3349 ; 7944
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:liu:diva-98066 (URN)10.1007/978-3-642-38886-6_45 (DOI)000342988500045 ()978-3-642-38885-9 (ISBN)978-3-642-38886-6 (ISBN)
Conference
18th Scandinavian Conferences on Image Analysis (SCIA 2013), 17-20 June 2013, Espoo, Finland.
Projects
CIMSMAP
Available from: 2013-09-27 Created: 2013-09-27 Last updated: 2019-04-12Bibliographically approved
3. Highly Accurate Attitude Estimation via Horizon Detection
Open this publication in new window or tab >>Highly Accurate Attitude Estimation via Horizon Detection
2016 (English)In: Journal of Field Robotics, ISSN 1556-4959, E-ISSN 1556-4967, Vol. 33, no 7, p. 967-993Article in journal (Refereed) Published
Abstract [en]

Attitude (pitch and roll angle) estimation from visual information is necessary for GPS-free navigation of airborne vehicles. We propose a highly accurate method to estimate the attitude by horizon detection in fisheye images. A Canny edge detector and a probabilistic Hough voting scheme are used to compute an approximate attitude and the corresponding horizon line in the image. Horizon edge pixels are extracted in a band close to the approximate horizon line. The attitude estimates are refined through registration of the extracted edge pixels with the geometrical horizon from a digital elevation map (DEM), in our case the SRTM3 database, extracted at a given approximate position. The proposed method has been evaluated using 1629 images from a flight trial with flight altitudes up to 600 m in an area with ground elevations ranging from sea level up to 500 m. Compared with the ground truth from a filtered inertial measurement unit (IMU)/GPS solution, the standard deviation for the pitch and roll angle errors obtained with 30 Mpixel images are 0.04° and 0.05°, respectively, with mean errors smaller than 0.02°. To achieve the high-accuracy attitude estimates, the ray refraction in the earth's atmosphere has been taken into account. The attitude errors obtained on real images are less or equal to those achieved on synthetic images for previous methods with DEM refinement, and the errors are about one order of magnitude smaller than for any previous vision-based method without DEM refinement.

Place, publisher, year, edition, pages
John Wiley & Sons, 2016
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:liu:diva-108212 (URN)10.1002/rob.21639 (DOI)000387925400005 ()
Note

At the date of the thesis presentation was this publication a manuscript.

Funding agencies: Swedish Governmental Agency for Innovation Systems, VINNOVA [NFFP5 2013-05243]; Swedish Foundation for Strategic Research [RIT10-0047]; Swedish Research Council within the Linnaeus environment CADICS; Knut and Alice Wallenberg Foundation

Available from: 2014-06-26 Created: 2014-06-26 Last updated: 2019-04-12Bibliographically approved
4. Improved Learning in Convolutional Neural Networks with Shifted Exponential Linear Units (ShELUs)
Open this publication in new window or tab >>Improved Learning in Convolutional Neural Networks with Shifted Exponential Linear Units (ShELUs)
2018 (English)In: 2018 24th International Conference on Pattern Recognition (ICPR), IEEE, 2018, p. 517-522Conference paper, Published paper (Refereed)
Abstract [en]

The Exponential Linear Unit (ELU) has been proven to speed up learning and improve the classification performance over activation functions such as ReLU and Leaky ReLU for convolutional neural networks. The reasons behind the improved behavior are that ELU reduces the bias shift, it saturates for large negative inputs and it is continuously differentiable. However, it remains open whether ELU has the optimal shape and we address the quest for a superior activation function.We use a new formulation to tune a piecewise linear activation function during training, to investigate the above question, and learn the shape of the locally optimal activation function. With this tuned activation function, the classification performance is improved and the resulting, learned activation function shows to be ELU-shaped irrespective if it is initialized as a RELU, LReLU or ELU. Interestingly, the learned activation function does not exactly pass through the origin indicating that a shifted ELU-shaped activation function is preferable. This observation leads us to introduce the Shifted Exponential Linear Unit (ShELU) as a new activation function.Experiments on Cifar-100 show that the classification performance is further improved when using the ShELU activation function in comparison with ELU. The improvement is achieved when learning an individual bias shift for each neuron.

Place, publisher, year, edition, pages
IEEE, 2018
Series
International Conference on Pattern Recognition
Keywords
CNN, activation function
National Category
Other Computer and Information Science Computer Vision and Robotics (Autonomous Systems)
Identifiers
urn:nbn:se:liu:diva-151606 (URN)10.1109/ICPR.2018.8545104 (DOI)000455146800087 ()978-1-5386-3787-6 (ISBN)
Conference
24th International Conference on Pattern Recognition, ICPR 2018, Beijing, China, 20-24 Aug. 2018
Funder
Wallenberg Foundations
Note

Funding agencies:  Wallenberg AI, Autonomous Systems and Software Program (WASP) - Knut and Alice Wallenberg Foundation; Swedish Research Council [2014-6227]

Available from: 2018-09-27 Created: 2018-09-27 Last updated: 2020-02-03

Open Access in DiVA

Vision-based Localization and Attitude Estimation Methods in Natural Environments(44058 kB)2465 downloads
File information
File name FULLTEXT01.pdfFile size 44058 kBChecksum SHA-512
587f5af303e86116dfcd9fd1549999241cf14daec5becd02942accf1937f5dc9cf5eb1d0942e0bbb04cc79a97309d1062c3ec6e81de29de729e891df29bfd88c
Type fulltextMimetype application/pdf
errata(123 kB)91 downloads
File information
File name ERRATA01.pdfFile size 123 kBChecksum SHA-512
59747be841832eb457eed5703e0a41973b08127cdc2c6cdd69654da8044c9c840a35933bddb5c2155100edb0c63551676359f31926b567ca63b5c5ebb62a3b7b
Type errataMimetype application/pdf

Other links

Publisher's full text

Authority records

Grelsson, Bertil

Search in DiVA

By author/editor
Grelsson, Bertil
By organisation
Computer VisionFaculty of Science & Engineering
Computer Vision and Robotics (Autonomous Systems)

Search outside of DiVA

GoogleGoogle Scholar
Total: 2466 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 2875 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf