liu.seSearch for publications in DiVA
Change search
ReferencesLink to record
Permanent link

Direct link
Regression density estimation with variational methods and stochastic approximation
National University of Singapore.
National University of Singapore.
Linköping University, Department of Computer and Information Science, Statistics. Linköping University, The Institute of Technology.
University of New South Wales, Sydney, Australia.
2012 (English)In: Journal of Computational And Graphical Statistics, ISSN 1061-8600, E-ISSN 1537-2715, Vol. 21, no 3, 797-820 p.Article in journal (Refereed) Published
Abstract [en]

Regression density estimation is the problem of flexibly estimating a response distribution as a function of covariates. An important approach to regression density estimation uses finite mixture models and our article considers flexible mixtures of heteroscedastic regression (MHR) models where the response distribution is a normal mixture, with the component means, variances and mixture weights all varying as a function of covariates. Our article develops fast variational approximation methods for inference. Our motivation is that alternative computationally intensive MCMC methods for fitting mixture models are difficult to apply when it is desired to fit models repeatedly in exploratory analysis and model choice. Our article makes three contributions. First, a variational approximation for MHR models is described where the variational lower bound is in closed form. Second, the basic approximation can be improved by using stochastic approximation methods to perturb the initial solution to attain higher accuracy. Third, the advantages of our approach for model choice and evaluation compared to MCMC based approaches are illustrated. These advantages are particularly compelling for time series data where repeated refitting for one step ahead prediction in model choice and diagnostics and in rolling window computations is very common. Supplemental materials for the article are available online.

Place, publisher, year, edition, pages
Taylor & Francis, 2012. Vol. 21, no 3, 797-820 p.
Keyword [en]
Bayesian model selection, heteroscedasticity, mixtures of experts, stochastic approximation, variational approximation
National Category
Probability Theory and Statistics
URN: urn:nbn:se:liu:diva-79583DOI: 10.1080/10618600.2012.679897ISI: 000308282000013OAI: diva2:543809

funding agencies|Singapore Ministry of Education (MOE)|R-155-000-068-133|Singapore Delft Water Alliances (SDWA) tropical reservoir research program||Australian Research Council (ARC)|DP0988579|

Available from: 2012-08-10 Created: 2012-08-10 Last updated: 2012-11-28

Open Access in DiVA

fulltext(1741 kB)696 downloads
File information
File name FULLTEXT01.pdfFile size 1741 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Search in DiVA

By author/editor
Villani, Mattias
By organisation
StatisticsThe Institute of Technology
In the same journal
Journal of Computational And Graphical Statistics
Probability Theory and Statistics

Search outside of DiVA

GoogleGoogle Scholar
Total: 696 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 94 hits
ReferencesLink to record
Permanent link

Direct link