liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Analytical Approximations for Bayesian Inference
Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Faculty of Science & Engineering. (Automatic Control)
2015 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Bayesian inference is a statistical inference technique in which Bayes’ theorem is used to update the probability distribution of a random variable using observations. Except for few simple cases, expression of such probability distributions using compact analytical expressions is infeasible. Approximation methods are required to express the a priori knowledge about a random variable in form of prior distributions. Further approximations are needed to compute posterior distributions of the random variables using the observations. When the computational complexity of representation of such posteriors increases over time as in mixture models, approximations are required to reduce the complexity of such representations.

This thesis further extends existing approximation methods for Bayesian inference, and generalizes the existing approximation methods in three aspects namely; prior selection, posterior evaluation given the observations and maintenance of computation complexity.

Particularly, the maximum entropy properties of the first-order stable spline kernel for identification of linear time-invariant stable and causal systems are shown. Analytical approximations are used to express the prior knowledge about the properties of the impulse response of a linear time-invariant stable and causal system.

Variational Bayes (VB) method is used to compute an approximate posterior in two inference problems. In the first problem, an approximate posterior for the state smoothing problem for linear statespace models with unknown and time-varying noise covariances is proposed. In the second problem, the VB method is used for approximate inference in state-space models with skewed measurement noise.

Moreover, a novel approximation method for Bayesian inference is proposed. The proposed Bayesian inference technique is based on Taylor series approximation of the logarithm of the likelihood function. The proposed approximation is devised for the case where the prior distribution belongs to the exponential family of distributions.

Finally, two contributions are dedicated to the mixture reduction (MR) problem. The first contribution, generalize the existing MR algorithms for Gaussian mixtures to the exponential family of distributions and compares them in an extended target tracking scenario. The second contribution, proposes a new Gaussian mixture reduction algorithm which minimizes the reverse Kullback-Leibler divergence and has specific peak preserving properties.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2015. , 79 p.
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 1710
National Category
Signal Processing
Identifiers
URN: urn:nbn:se:liu:diva-121619DOI: 10.3384/diss.diva-121619ISBN: 978-91-7685-930-8 (print)OAI: oai:DiVA.org:liu-121619DiVA: diva2:858322
Public defence
2015-11-06, Visionen, B-huset, Campus Valla, Linköping, 10:15 (English)
Opponent
Supervisors
Available from: 2015-10-05 Created: 2015-09-28 Last updated: 2015-10-07Bibliographically approved
List of papers
1. Maximum entropy properties of discrete-time first-order stable spline kernel
Open this publication in new window or tab >>Maximum entropy properties of discrete-time first-order stable spline kernel
Show others...
2016 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 66, 34-38 p.Article in journal (Refereed) Published
Abstract [en]

The first order stable spline (SS-1) kernel (also known as the tunedcorrelated kernel) is used extensively in regularized system identification, where the impulse response is modeled as a zero-mean Gaussian process whose covariance function is given by well designed and tuned kernels. In this paper, we discuss the maximum entropy properties of this kernel. In particular, we formulate the exact maximum entropy problem solved by the SS-1 kernel without Gaussian and uniform sampling assumptions. Under general sampling assumption, we also derive the special structure of the SS-1 kernel (e.g. its tridiagonal inverse and factorization have closed form expression), also giving to it a maximum entropy covariance completion interpretation.

Keyword
System identification;Regularization method;Kernel structure;Maximum entropy
National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-121618 (URN)10.1016/j.automatica.2015.12.009 (DOI)
Available from: 2015-09-28 Created: 2015-09-28 Last updated: 2017-12-01Bibliographically approved
2. Approximate Bayesian Smoothing with Unknown Process and Measurement Noise Covariances
Open this publication in new window or tab >>Approximate Bayesian Smoothing with Unknown Process and Measurement Noise Covariances
2015 (English)In: IEEE Signal Processing Letters, ISSN 1070-9908, E-ISSN 1558-2361, Vol. 22, no 12, 2450-2454 p.Article in journal (Refereed) Published
Abstract [en]

We present an adaptive smoother for linear state-space models with unknown process and measurement noise covariances. The proposed method utilizes the variational Bayes technique to perform approximate inference. The resulting smoother is computationally efficient, easy to implement, and can be applied to high dimensional linear systems. The performance of the algorithm is illustrated on a target tracking example.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2015
Keyword
Adaptive smoothing, Kalman filtering, noise covariance, Rauch-Tung-Striebel smoother, sensor calibration, time-varying noiseco variances, variational Bayes
National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-121617 (URN)10.1109/LSP.2015.2490543 (DOI)
Note

At the time for thesis presentation publication was in status: Manuscript

Available from: 2015-09-28 Created: 2015-09-28 Last updated: 2017-12-01Bibliographically approved
3. Robust Inference for State-Space Models with Skewed Measurement Noise
Open this publication in new window or tab >>Robust Inference for State-Space Models with Skewed Measurement Noise
2015 (English)In: IEEE Signal Processing Letters, ISSN 1070-9908, E-ISSN 1558-2361, Vol. 22, no 11, 1898-1902 p.Article in journal (Refereed) Published
Abstract [en]

Filtering and smoothing algorithms for linear discrete-time state-space models with skewed and heavy-tailed measurement noise are presented. The algorithms use a variational Bayes approximation of the posterior distribution of models that have normal prior and skew-t-distributed measurement noise. The proposed filter and smoother are compared with conventional low-complexity alternatives in a simulated pseudorange positioning scenario. In the simulations the proposed methods achieve better accuracy than the alternative methods, the computational complexity of the filter being roughly 5 to 10 times that of the Kalman filter.

Place, publisher, year, edition, pages
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, 2015
Keyword
Kalman filter; robust filtering; RTS smoother; skew t; skewness; t-distribution; variational Bayes
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:liu:diva-120129 (URN)10.1109/LSP.2015.2437456 (DOI)000356458700003 ()
Note

Funding Agencies|Tampere University of Technology Graduate School; Finnish Doctoral Programme in Computational Sciences (FICS); Foundation of Nokia Corporation; Swedish research council (VR), project ETT [621-2010-4301]

Available from: 2015-07-14 Created: 2015-07-13 Last updated: 2017-12-04
4. Bayesian Inference via Approximation of Log-likelihood for Priors in Exponential Family
Open this publication in new window or tab >>Bayesian Inference via Approximation of Log-likelihood for Priors in Exponential Family
(English)Manuscript (preprint) (Other academic)
Abstract [en]

In this paper, a Bayesian inference technique based on Taylor series approximation of the logarithm of the likelihood function is presented. The proposed approximation is devised for the case where the prior distribution belongs to the exponential family of distributions. The logarithm of the likelihood function is linearized with respect to the sufficient statistic of the prior distribution in exponential family such that the posterior obtains the same exponential family form as the prior. Similarities between the proposed method and the extended Kalman filter for nonlinear filtering are illustrated. Further, an extended target measurement update for target models where the target extent is represented by a random matrix having an inverse Wishart distribution is derived. The approximate update covers the important case where the spread of measurement is due to the target extent as well as the measurement noise in the sensor.

National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-121616 (URN)
Available from: 2015-09-28 Created: 2015-09-28 Last updated: 2015-10-05Bibliographically approved
5. Greedy Reduction Algorithms for Mixtures of Exponential Family
Open this publication in new window or tab >>Greedy Reduction Algorithms for Mixtures of Exponential Family
2015 (English)In: IEEE Signal Processing Letters, ISSN 1070-9908, E-ISSN 1558-2361, Vol. 22, no 6, 676-680 p.Article in journal (Refereed) Published
Abstract [en]

In this letter, we propose a general framework for greedy reduction of mixture densities of exponential family. The performances of the generalized algorithms are illustrated both on an artificial example where randomly generated mixture densities are reduced and on a target tracking scenario where the reduction is carried out in the recursion of a Gaussian inverse Wishart probability hypothesis density (PHD) filter.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2015
Keyword
Exponential family; extended target; integral square error; Kullback-Leibler divergence; mixture density; mixture reduction; target tracking
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:liu:diva-112990 (URN)10.1109/LSP.2014.2367154 (DOI)000345236400005 ()
Note

Funding Agencies|Swedish research council (VR) under ETT [621-2010-4301]; SSF, project CUAS

Available from: 2015-01-12 Created: 2015-01-08 Last updated: 2017-12-05
6. Gaussian Mixture Reduction Using Reverse Kullback-Leibler Divergence
Open this publication in new window or tab >>Gaussian Mixture Reduction Using Reverse Kullback-Leibler Divergence
(English)Manuscript (preprint) (Other academic)
National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-121615 (URN)
Available from: 2015-09-28 Created: 2015-09-28 Last updated: 2015-10-05

Open Access in DiVA

fulltext(1512 kB)623 downloads
File information
File name FULLTEXT02.pdfFile size 1512 kBChecksum SHA-512
54a0ea628714bdf8e10a377cf31ce51a436522e4d528ccc66979b5a5f7b19560f1524e86219f2ff2b35c0279b21312af0877da113be446df56bcbfd1d01740d8
Type fulltextMimetype application/pdf
omslag(21 kB)30 downloads
File information
File name COVER01.pdfFile size 21 kBChecksum SHA-512
3b46b7f76dd8be8a1335d98bd8efe9de8061c54eeb5d1e6bf40b4cfebb9470c8e40b35f6cb9fac13a1689c21cefcb54b9406a788d43b0bbbe3bb1471eec0e62a
Type coverMimetype application/pdf

Other links

Publisher's full text

Authority records BETA

Ardeshiri, Tohid

Search in DiVA

By author/editor
Ardeshiri, Tohid
By organisation
Automatic ControlFaculty of Science & Engineering
Signal Processing

Search outside of DiVA

GoogleGoogle Scholar
Total: 637 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 3493 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf