liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Nested Sequential Monte Carlo Methods
Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Faculty of Science & Engineering.
The University of Cambridge, Cambridge, United Kingdom.
Uppsala University, Uppsala, Sweden.
2015 (English)In: Proceedings of The 32nd International Conference on Machine Learning / [ed] Francis Bach, David Blei, Journal of Machine Learning Research (Online) , 2015, Vol. 37, p. 1292-1301Conference paper, Published paper (Refereed)
Abstract [en]

We propose nested sequential Monte Carlo (NSMC), a methodology to sample from sequences of probability distributions, even where the random variables are high-dimensional. NSMC generalises the SMC framework by requiring only approximate, properly weighted, samples from the SMC proposal distribution, while still resulting in a correct SMC algorithm. Furthermore, NSMC can in itself be used to produce such properly weighted samples. Consequently, one NSMC sampler can be used to construct an efficient high-dimensional proposal distribution for another NSMC sampler, and this nesting of the algorithm can be done to an arbitrary degree. This allows us to consider complex and high-dimensional models using SMC. We show results that motivate the efficacy of our approach on several filtering problems with dimensions in the order of 100 to 1 000.

Place, publisher, year, edition, pages
Journal of Machine Learning Research (Online) , 2015. Vol. 37, p. 1292-1301
Series
JMLR Workshop and Conference Proceedings, ISSN 1938-7228 ; 37
National Category
Computer Sciences Control Engineering Probability Theory and Statistics
Identifiers
URN: urn:nbn:se:liu:diva-122698OAI: oai:DiVA.org:liu-122698DiVA, id: diva2:871698
Conference
32nd International Conference on Machine Learning, Lille, France, 6-11 July, 2015
Available from: 2015-11-16 Created: 2015-11-16 Last updated: 2018-11-09Bibliographically approved
In thesis
1. Machine learning using approximate inference: Variational and sequential Monte Carlo methods
Open this publication in new window or tab >>Machine learning using approximate inference: Variational and sequential Monte Carlo methods
2018 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Automatic decision making and pattern recognition under uncertainty are difficult tasks that are ubiquitous in our everyday life. The systems we design, and technology we develop, requires us to coherently represent and work with uncertainty in data. Probabilistic models and probabilistic inference gives us a powerful framework for solving this problem. Using this framework, while enticing, results in difficult-to-compute integrals and probabilities when conditioning on the observed data. This means we have a need for approximate inference, methods that solves the problem approximately using a systematic approach. In this thesis we develop new methods for efficient approximate inference in probabilistic models.

There are generally two approaches to approximate inference, variational methods and Monte Carlo methods. In Monte Carlo methods we use a large number of random samples to approximate the integral of interest. With variational methods, on the other hand, we turn the integration problem into that of an optimization problem. We develop algorithms of both types and bridge the gap between them.

First, we present a self-contained tutorial to the popular sequential Monte Carlo (SMC) class of methods. Next, we propose new algorithms and applications based on SMC for approximate inference in probabilistic graphical models. We derive nested sequential Monte Carlo, a new algorithm particularly well suited for inference in a large class of high-dimensional probabilistic models. Then, inspired by similar ideas we derive interacting particle Markov chain Monte Carlo to make use of parallelization to speed up approximate inference for universal probabilistic programming languages. After that, we show how we can make use of the rejection sampling process when generating gamma distributed random variables to speed up variational inference. Finally, we bridge the gap between SMC and variational methods by developing variational sequential Monte Carlo, a new flexible family of variational approximations.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2018. p. 39
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 1969
National Category
Control Engineering Computer Sciences Signal Processing
Identifiers
urn:nbn:se:liu:diva-152647 (URN)10.3384/diss.diva-152647 (DOI)9789176851616 (ISBN)
Public defence
2018-12-14, Ada Lovelace, Building B, Campus Valla, Linköping, 10:15 (English)
Opponent
Supervisors
Available from: 2018-11-27 Created: 2018-11-09 Last updated: 2018-12-05Bibliographically approved

Open Access in DiVA

fulltext(900 kB)121 downloads
File information
File name FULLTEXT02.pdfFile size 900 kBChecksum SHA-512
ff8307b5b8d86ef6634a067cf571933cf7eab44f300316fd80b706d0b25d37cfd4d5e4c4b98bacd077c59d387189524dfc2635b00bf5176264b9e283c71f335d
Type fulltextMimetype application/pdf

Other links

Link to publication

Authority records BETA

Andersson Naesseth, ChristianLindsten, Fredrik

Search in DiVA

By author/editor
Andersson Naesseth, ChristianLindsten, Fredrik
By organisation
Automatic ControlFaculty of Science & Engineering
Computer SciencesControl EngineeringProbability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 127 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 1081 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf