liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
P-SGLD: Stochastic Gradient Langevin Dynamics with control variates
Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning.
2017 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

Year after years, the amount of data that we continuously generate is increasing. When this situation started the main challenge was to find a way to store the huge quantity of information. Nowadays, with the increasing availability of storage facilities, this problem is solved but it gives us a new issue to deal with: find tools that allow us to learn from this large data sets. In this thesis, a framework for Bayesian learning with the ability to scale to large data sets is studied. We present the Stochastic Gradient Langevin Dynamics (SGLD) framework and show that in some cases its approximation of the posterior distribution is quite poor. A reason for this can be that SGLD estimates the gradient of the log-likelihood with a high variability due to naïve sampling. Our approach combines accurate proxies for the gradient of the log-likelihood with SGLD. We show that it produces better results in terms of convergence to the correct posterior distribution than the standard SGLD, since accurate proxies dramatically reduce the variance of the gradient estimator. Moreover, we demonstrate that this approach is more efficient than the standard Markov Chain Monte Carlo (MCMC) method and that it exceeds other techniques of variance reduction proposed in the literature such as SAGA-LD algorithm. This approach also uses control variates to improve SGLD so that it is straightforward the comparison with our approach. We apply the method to the Logistic Regression model. 

Place, publisher, year, edition, pages
2017. , p. 40
Series
Linköping Electronic Articles in Computer and Information Science, ISSN 1401-9841
Keywords [en]
Big Data, Bayesian Inference, MCMC, SGLD, Estimated Gradient, Logistic Regression
National Category
Probability Theory and Statistics
Identifiers
URN: urn:nbn:se:liu:diva-140121ISRN: LIU-IDA/STAT-A--17/004-SEOAI: oai:DiVA.org:liu-140121DiVA, id: diva2:1137253
Subject / course
Statistics
Supervisors
Examiners
Available from: 2017-08-30 Created: 2017-08-30 Last updated: 2017-08-30Bibliographically approved

Open Access in DiVA

fulltext(12075 kB)180 downloads
File information
File name FULLTEXT01.pdfFile size 12075 kBChecksum SHA-512
e8ca3f49224344db28b8abfc6cea6e565c55daa5217a9575593fffb58aaf2fc1b01d186cb1f12d9d5a64a950e7ec8822297ef090810794c4440ef49eba40441c
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Bruzzone, Andrea
By organisation
The Division of Statistics and Machine Learning
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 180 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 477 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf