liu.seSearch for publications in DiVA
Change search
Refine search result
12 1 - 50 of 70
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Ahmadian, Amirhossein
    et al.
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Science & Engineering.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Science & Engineering.
    Enhancing Representation Learning with Deep Classifiers in Presence of Shortcut2023In: Proceedings of IEEE ICASSP 2023, 2023Conference paper (Refereed)
    Abstract [en]

    A deep neural classifier trained on an upstream task can be leveraged to boost the performance of another classifier in a related downstream task through the representations learned in hidden layers. However, presence of shortcuts (easy-to-learn features) in the upstream task can considerably impair the versatility of intermediate representations and, in turn, the downstream performance. In this paper, we propose a method to improve the representations learned by deep neural image classifiers in spite of a shortcut in upstream data. In our method, the upstream classification objective is augmented with a type of adversarial training where an auxiliary network, so called lens, fools the classifier by exploiting the shortcut in reconstructing images. Empirical comparisons in self-supervised and transfer learning problems with three shortcut-biased datasets suggest the advantages of our method in terms of downstream performance and/or training time.

  • 2.
    Ahmadian, Amirhossein
    et al.
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Science & Engineering.
    Lindsten, Fredrik
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Science & Engineering.
    Likelihood-free Out-of-Distribution Detection with Invertible Generative Models2021In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI 2021), 2021Conference paper (Refereed)
    Abstract [en]

    Likelihood of generative models has been used traditionally as a score to detect atypical (Out-of-Distribution, OOD) inputs. However, several recent studies have found this approach to be highly unreliable, even with invertible generative models, where computing the likelihood is feasible. In this paper, we present a different framework for generative model--based OOD detection that employs the model in constructing a new representation space, instead of using it directly in computing typicality scores, where it is emphasized that the score function should be interpretable as the similarity between the input and training data in the new space. In practice, with a focus on invertible models, we propose to extract low-dimensional features (statistics) based on the model encoder and complexity of input images, and then use a One-Class SVM to score the data. Contrary to recently proposed OOD detection methods for generative models, our method does not require computing likelihood values. Consequently, it is much faster when using invertible models with iteratively approximated likelihood (e.g. iResNet), while it still has a performance competitive with other related methods

  • 3.
    Andersson Naesseth, Christian
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Capacity estimation of two-dimensional channels using Sequential Monte Carlo2014In: 2014 IEEE Information Theory Workshop, 2014, p. 431-435Conference paper (Refereed)
    Abstract [en]

    We derive a new Sequential-Monte-Carlo-based algorithm to estimate the capacity of two-dimensional channel models. The focus is on computing the noiseless capacity of the 2-D (1, ∞) run-length limited constrained channel, but the underlying idea is generally applicable. The proposed algorithm is profiled against a state-of-the-art method, yielding more than an order of magnitude improvement in estimation accuracy for a given computation time.

    Download full text (pdf)
    fulltext
  • 4.
    Andersson Naesseth, Christian
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Faculty of Science & Engineering.
    Lindsten, Fredrik
    The University of Cambridge, Cambridge, United Kingdom.
    Schön, Thomas
    Uppsala University, Uppsala, Sweden.
    Nested Sequential Monte Carlo Methods2015In: Proceedings of The 32nd International Conference on Machine Learning / [ed] Francis Bach, David Blei, Journal of Machine Learning Research (Online) , 2015, Vol. 37, p. 1292-1301Conference paper (Refereed)
    Abstract [en]

    We propose nested sequential Monte Carlo (NSMC), a methodology to sample from sequences of probability distributions, even where the random variables are high-dimensional. NSMC generalises the SMC framework by requiring only approximate, properly weighted, samples from the SMC proposal distribution, while still resulting in a correct SMC algorithm. Furthermore, NSMC can in itself be used to produce such properly weighted samples. Consequently, one NSMC sampler can be used to construct an efficient high-dimensional proposal distribution for another NSMC sampler, and this nesting of the algorithm can be done to an arbitrary degree. This allows us to consider complex and high-dimensional models using SMC. We show results that motivate the efficacy of our approach on several filtering problems with dimensions in the order of 100 to 1 000.

    Download full text (pdf)
    fulltext
  • 5.
    Andersson Naesseth, Christian
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Lindsten, Fredrik
    University of Cambridge, Cambridge, UK.
    Schön, Thomas
    Uppsala University, Uppsala, Sweden.
    Sequential Monte Carlo for Graphical Models2014In: Advances in Neural Information Processing Systems, 2014, p. 1862-1870Conference paper (Refereed)
    Abstract [en]

    We propose a new framework for how to use sequential Monte Carlo (SMC) algorithms for inference in probabilistic graphical models (PGM). Via a sequential decomposition of the PGM we find a sequence of auxiliary distributions defined on a monotonically increasing sequence of probability spaces. By targeting these auxiliary distributions using SMC we are able to approximate the full joint distribution defined by the PGM. One of the key merits of the SMC sampler is that it provides an unbiased estimate of the partition function of the model. We also show how it can be used within a particle Markov chain Monte Carlo framework in order to construct high-dimensional block-sampling algorithms for general PGMs.

    Download full text (pdf)
    fulltext
  • 6.
    Calafat, Francisco M.
    et al.
    Natl Oceanog Ctr, Joseph Proudman Bldg,6 Brownlow St, Liverpool L3 5DA, Merseyside, England.
    Wahl, Thomas
    Univ Cent Florida, Natl Ctr Integrated Coastal Res, 12800 Pegasus Dr,Suite 211, Orlando, FL 32816 USA;Univ Cent Florida, Dept Civil Environm & Construct Engn, USA.
    Lindsten, Fredrik
    Uppsala universitet, Reglerteknik, Sweden.
    Williams, Joanne
    Natl Oceanog Ctr, England.
    Frajka-Williams, Eleanor
    Univ Southampton, Ocean & Earth Sci, European Way, England.
    Coherent modulation of the sea-level annual cycle in the United States by Atlantic Rossby waves2018In: Nature Communications, E-ISSN 2041-1723, Vol. 9, article id 2571Article in journal (Refereed)
    Abstract [en]

    Changes in the sea-level annual cycle (SLAC) can have profound impacts on coastal areas, including increased flooding risk and ecosystem alteration, yet little is known about the magnitude and drivers of such changes. Here we show, using novel Bayesian methods, that there are significant decadal fluctuations in the amplitude of the SLAC along the United States Gulf and Southeast coasts, including an extreme event in 2008-2009 that is likely (probability = 68%) unprecedented in the tide-gauge record. Such fluctuations are coherent along the coast but decoupled from deep-ocean changes. Through the use of numerical and analytical ocean models, we show that the primary driver of these fluctuations involves incident Rossby waves that generate fast western-boundary waves. These Rossby waves project onto the basin-wide upper mid-ocean transport (top 1000 m) leading to a link with the SLAC, wherein larger SLAC amplitudes coincide with enhanced transport variability.

    Download full text (pdf)
    FULLTEXT01
  • 7.
    Dahlin, Johan
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Particle filter-based Gaussian process optimisation for parameter inference2014In: Proceedings of the 19th IFAC World Congress, 2014 / [ed] Edward Boje and Xiaohua Xia, 2014, p. 8675-8680Conference paper (Refereed)
    Abstract [en]

    We propose a novel method for maximum-likelihood-based parameter inference in nonlinear and/or non-Gaussian state space models. The method is an iterative procedure with three steps. At each iteration a particle filter is used to estimate the value of the log-likelihood function at the current parameter iterate. Using these log-likelihood estimates, a surrogate objective function is created by utilizing a Gaussian process model. Finally, we use a heuristic procedure to obtain a revised parameter iterate, providing an automatic trade-off between exploration and exploitation of the surrogate model. The method is profiled on two state space models with good performance both considering accuracy and computational cost.

  • 8.
    Dahlin, Johan
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Particle Metropolis Hastings using Langevin Dynamics2013In: Proceedings of the 38th International Conference on Acoustics, Speech, and Signal Processing, IEEE conference proceedings, 2013, p. 6308-6312Conference paper (Refereed)
    Abstract [en]

    Particle Markov Chain Monte Carlo (PMCMC) samplers allow for routine inference of parameters and states in challenging nonlinear problems. A common choice for the parameter proposal is a simple random walk sampler, which can scale poorly with the number of parameters.

    In this paper, we propose to use log-likelihood gradients, i.e. the score, in the construction of the proposal, akin to the Langevin Monte Carlo method, but adapted to the PMCMC framework. This can be thought of as a way to guide a random walk proposal by using drift terms that are proportional to the score function. The method is successfully applied to a stochastic volatility model and the drift term exhibits intuitive behaviour.

    Download full text (pdf)
    fulltext
  • 9.
    Dahlin, Johan
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Dept. of Information Technology, Uppsala University, Uppsala, Sweden.
    Particle Metropolis-Hastings using gradient and Hessian information2015In: Statistics and computing, ISSN 0960-3174, E-ISSN 1573-1375, Vol. 25, no 1, p. 81-92Article in journal (Refereed)
    Abstract [en]

    Particle Metropolis-Hastings (PMH) allows for Bayesian parameter inference in nonlinear state space models by combining MCMC and particle filtering. The latter is used to estimate the intractable likelihood. In its original formulation, PMH makes use of a marginal MCMC proposal for the parameters, typically a Gaussian random walk. However, this can lead to a poor exploration of the parameter space and an inefficient use of the generated particles.

    We propose two alternative versions of PMH that incorporate gradient and Hessian information about the posterior into the proposal. This information is more or less obtained as a byproduct of the likelihood estimation. Indeed, we show how to estimate the required information using a fixed-lag particle smoother, with a computational cost growing linearly in the number of particles. We conclude that the proposed methods can: (i) decrease the length of the burn-in phase, (ii) increase the mixing of the Markov chain at the stationary phase, and (iii) make the proposal distribution scale invariant which simplifies tuning.

    Download full text (pdf)
    fulltext
  • 10.
    Dahlin, Johan
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology. Uppsala University.
    Second-Order Particle MCMC for Bayesian Parameter Inference2014In: Proceedings of the 19th IFAC World Congress, 2014, p. 8656-8661Conference paper (Refereed)
    Abstract [en]

    We propose an improved proposal distribution in the Particle Metropolis-Hastings (PMH) algorithm for Bayesian parameter inference in nonlinear state space models. This proposal incorporates second-order information about the parameter posterior distribution, which can be extracted from the particle filter already used within the PMH algorithm. The added information makes the proposal scale-invariant, simpler to tune and can possibly also shorten the burn-in phase. The proposed algorithm has a computational cost which is proportional to the number of particles, i.e. the same as the original marginal PMH algorithm. Finally, we provide two numerical examples that illustrates some of the possible benefits of adding the second-order information.

  • 11.
    Dahlin, Johan
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas Bo
    Dept. of Information Technology, Uppsala University, Uppsala, Sweden.
    Wills, Adrian George
    School of EECS, University of Newcastle, Australia .
    Hierarchical Bayesian approaches for robust inference in ARX models2012In: Proceedings from the 16th IFAC Symposium on System Identification, 2012 / [ed] Michel Kinnaert, International Federation of Automatic Control , 2012, Vol. 16 Part 1, p. 131-136Conference paper (Refereed)
    Abstract [en]

    Gaussian innovations are the typical choice in most ARX models but using other distributions such as the Student's t could be useful. We demonstrate that this choice of distribution for the innovations provides an increased robustness to data anomalies, such as outliers and missing observations. We consider these models in a Bayesian setting and perform inference using numerical procedures based on Markov Chain Monte Carlo methods. These models include automatic order determination by two alternative methods, based on a parametric model order and a sparseness prior, respectively. The methods and the advantage of our choice of innovations are illustrated in three numerical studies using both simulated data and real EEG data.

    Download full text (pdf)
    fulltext
  • 12.
    Dahlin, Johan
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Wills, Adrian
    University of Newcastle, Australia.
    Robust ARX Models with Automatic Order Determination and Student's t Innovations2011Report (Other academic)
    Abstract [en]

    ARX models is a common class of models of dynamical systems. Here, we consider the case when the innovation process is not well described by Gaussian noise and instead propose to model the driving noise as Student's t distributed. The t distribution is more heavy tailed than the Gaussian distribution, which provides an increased robustness to data anomalies, such as outliers and missing observations. We use a Bayesian setting and design the models to also include an automatic order determination. Basically, this means that we infer knowledge about the posterior distribution of the model order from data. We consider two related models, one with a parametric model order and one with a sparseness prior on the ARX coefficients. We derive Markov chain Monte Carlo samplers to perform inference in these models. Finally, we provide three numerical illustrations with both simulated data and real EEG data to evaluate the proposed methods.

    Download full text (pdf)
    FULLTEXT01
  • 13.
    Dahlin, Johan
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Faculty of Science & Engineering.
    Schön, Thomas Bo
    Department of Information Technology, Uppsala University.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Faculty of Science & Engineering.
    Quasi-Newton particle Metropolis-Hastings2015In: Proceedings of the 17th IFAC Symposium on System Identification., Elsevier, 2015, Vol. 48 Issue 28, p. 981-986Conference paper (Refereed)
    Abstract [en]

    Particle Metropolis-Hastings enables Bayesian parameter inference in general nonlinear state space models (SSMs). However, in many implementations a random walk proposal is used and this can result in poor mixing if not tuned correctly using tedious pilot runs. Therefore, we consider a new proposal inspired by quasi-Newton algorithms that may achieve similar (or better) mixing with less tuning. An advantage compared to other Hessian based proposals, is that it only requires estimates of the gradient of the log-posterior. A possible application is parameter inference in the challenging class of SSMs with intractable likelihoods.We exemplify this application and the benefits of the new proposal by modelling log-returns offuture contracts on coffee by a stochastic volatility model with alpha-stable observations.

    Download full text (pdf)
    fulltext
  • 14.
    Frigola, Roger
    et al.
    Department of Engineering, University of Cambridge, UK.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas B.
    Deptartment of Information Technology, Uppsala University, Sweden.
    Rasmussen, Carl E.
    Department of Engineering, University of Cambridge, UK.
    Bayesian Inference and Learning in Gaussian Process State-Space Models with Particle MCMC2013In: Advances in Neural Information Processing Systems 26 / [ed] C.J.C. Burges, L. Bottou, M. Welling, Z. Ghahramani and K.Q. Weinberger, 2013Conference paper (Refereed)
    Abstract [en]

    State-space models are successfully used in many areas of science, engineering and economics to model time series and dynamical systems. We present a fully Bayesian approach to inference and learning in nonlinear nonparametric state-space models. We place a Gaussian process prior over the transition dynamics, resulting in a flexible model able to capture complex dynamical phenomena. However, to enable efficient inference, we marginalize over the dynamics of the model and instead infer directly the joint smoothing distribution through the use of specially tailored Particle Markov Chain Monte Carlo samplers. Once an approximation of the smoothing distribution is computed, the state transition predictive distribution can be formulated analytically. We make use of sparse Gaussian process models to greatly reduce the computational complexity of the approach.

  • 15.
    Frigola, Roger
    et al.
    Department of Engineering, University of Cambridge, UK.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas B.
    Deptartment of Information Technology, Uppsala University, Sweden.
    Rasmussen, Carl E.
    Department of Engineering, University of Cambridge, UK.
    Identification of Gaussian process state-space models with particle stochastic approximation EM2014In: Proceedings of the 19th IFAC World Congress, Elsevier, 2014, Vol. 47Conference paper (Refereed)
    Abstract [en]

    Gaussian process state-space models (GP-SSMs) are a very exible family of models of nonlinear dynamical systems. They comprise a Bayesian nonparametric representation of the dynamics of the system and additional (hyper-)parameters governing the properties of this nonparametric representation. The Bayesian formalism enables systematic reasoning about the uncertainty in the system dynamics. We present an approach to maximum likelihood identification of the parameters in GP-SSMs, while retaining the full nonparametric description of the dynamics. The method is based on a stochastic approximation version of the EM algorithm that employs recent developments in particle Markov chain Monte Carlo for efficient identification.

    Download full text (pdf)
    fulltext
  • 16.
    Gunnarsson, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology. Ericsson Research, Linköping, Sweden.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Carlsson, N.
    Ericsson Research, Linköping, Sweden; Volvo Cars, Gothanburg, Sweden.
    Particle filtering for network-based positioning terrestrial radio networks2014In: Data Fusion & Target Tracking 2014: Algorithms and Applications (DF&TT 2014), IET Conference on, Institution of Engineering and Technology , 2014, Vol. 2014, no 629 CPConference paper (Refereed)
    Abstract [en]

    There is strong interest in positioing in wireless networks, partly to support end user service needs, but also to support network management with network-based network information. The focus in this paper is on the latter, while using measurements that are readily available in wireless networks. We show how thesignal direction of departure and inter-distance between the base station and the mobile terminal can be estimated, and how particle filters and smoothers can be used to post-process the measurements. The methods are evaluated in a live 3GPP LTE network with promising results inlcuding position error medians of less than 100 m.

  • 17.
    Jacob, Pierre
    et al.
    Harvard University, USA.
    Lindsten, Fredrik
    Uppsala universitet, Avdelningen för systemteknik, Sweden.
    Schön, Thomas B.
    Uppsala universitet, Avdelningen för systemteknik, Sweden.
    Retracted article: Smoothing with Couplings of Conditional Particle Filters2018In: Journal of the American Statistical Association, ISSN 0162-1459, E-ISSN 1537-274XArticle in journal (Refereed)
    Abstract [en]

    In state space models, smoothing refers to the task of estimating a latent stochastic process given noisy measurements related to the process. We propose an unbiased estimator of smoothing expectations. The lack-of-bias property has methodological benefits: independent estimators can be generated in parallel, and confidence intervals can be constructed from the central limit theorem to quantify the approximation error. To design unbiased estimators, we combine a generic debiasing technique for Markov chains, with a Markov chain Monte Carlo algorithm for smoothing. The resulting procedure is widely applicable and we show in numerical experiments that the removal of the bias comes at a manageable increase in variance. We establish the validity of the proposed estimators under mild assumptions. Numerical experiments are provided on toy models, including a setting of highly-informative observations, and for a realistic Lotka-Volterra model with an intractable transition density.

    Download full text (pdf)
    FULLTEXT01
  • 18.
    Lindholm, Andreas
    et al.
    Annotell, Göteborg, Sweden.
    Wahlström, Niklas
    Uppsala universitet, Sweden.
    Lindsten, Fredrik
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Science & Engineering.
    Schön, Thomas
    Uppsala universitet, Sweden.
    Machine learning: a first course for engineers and scientists2022Book (Other academic)
    Abstract [en]

    "This book introduces machine learning for readers with some background in basic linear algebra, statistics, probability, and programming. In a coherent statistical framework it covers a selection of supervised machine learning methods, from the most fundamental (k-NN, decision trees, linear and logistic regression) to more advanced methods (deep neural networks, support vector machines, Gaussian processes, random forests and boosting), plus commonly-used unsupervised methods (generative modeling, k-means, PCA, autoencoders and generative adversarial networks). Careful explanations and pseudo-code are presented for all methods. The authors maintain a focus on the fundamentals by drawing connections between methods and discussing general concepts such as loss functions, maximum likelihood, the bias-variance decomposition, ensemble averaging, kernels and the Bayesian approach along with generally useful tools such as regularization, cross validation, evaluation metrics and optimization methods. The final chapters offer practical advice for solving real-world supervised machine learning problems and on ethical aspects of modern machine learning"--

  • 19.
    Lindqvist, Jakob
    et al.
    Chalmers University of Technology, Department of Electrical Engineering, Gothenburg, Sweden.
    Olmin, Amanda
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Arts and Sciences.
    Lindsten, Fredrik
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Science & Engineering.
    Svensson, Lennart
    Chalmers University of Technology, Department of Electrical Engineering, Gothenburg, Sweden.
    A General Framework for Ensemble Distribution Distillation2020In: 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP), IEEE, 2020, p. 1-6Conference paper (Refereed)
    Abstract [en]

    Ensembles of neural networks have shown to give better predictive performance and more reliable uncertainty estimates than individual networks. Additionally, ensembles allow the uncertainty to be decomposed into aleatoric (data) and epistemic (model) components, giving a more complete picture of the predictive uncertainty. Ensemble distillation is the process of compressing an ensemble into a single model, often resulting in a leaner model that still outperforms the individual ensemble members. Unfortunately, standard distillation erases the natural uncertainty decomposition of the ensemble. We present a general framework for distilling both regression and classification ensembles in a way that preserves the decomposition. We demonstrate the desired behaviour of our framework and show that its predictive performance is on par with standard distillation.

    Download full text (pdf)
    fulltext
  • 20.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    An Efficient Stochastic Approximation EM Algorithm using Conditional Particle Filters2013In: Proceedings of the 38th International Conference on Acoustics, Speech, and Signal Processing, IEEE conference proceedings, 2013, p. 6274-6278Conference paper (Refereed)
    Abstract [en]

    I present a novel method for maximum likelihood parameter estimation in nonlinear/non-Gaussian state-space models. It is an expectation maximization (EM) like method, which uses sequential Monte Carlo (SMC) for the intermediate state inference problem. Contrary to existing SMC-based EM algorithms, however, it makes efficient use of the simulated particles through the use of particle Markov chain Monte Carlo (PMCMC) theory. More precisely, the proposed method combines the efficient conditional particle filter with ancestor sampling (CPF-AS) with the stochastic approximation EM (SAEM) algorithm. This results in a procedure which does not rely on asymptotics in the number of particles for convergence, meaning that the method is very computationally competitive. Indeed, the method is evaluated in a simulation study, using a small number of particles with promising results.

    Download full text (pdf)
    fulltext
  • 21. Order onlineBuy this publication >>
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Particle filters and Markov chains for learning of dynamical systems2013Doctoral thesis, comprehensive summary (Other academic)
    Abstract [en]

    Sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC) methods provide computational tools for systematic inference and learning in complex dynamical systems, such as nonlinear and non-Gaussian state-space models. This thesis builds upon several methodological advances within these classes of Monte Carlo methods.Particular emphasis is placed on the combination of SMC and MCMC in so called particle MCMC algorithms. These algorithms rely on SMC for generating samples from the often highly autocorrelated state-trajectory. A specific particle MCMC algorithm, referred to as particle Gibbs with ancestor sampling (PGAS), is suggested. By making use of backward sampling ideas, albeit implemented in a forward-only fashion, PGAS enjoys good mixing even when using seemingly few particles in the underlying SMC sampler. This results in a computationally competitive particle MCMC algorithm. As illustrated in this thesis, PGAS is a useful tool for both Bayesian and frequentistic parameter inference as well as for state smoothing. The PGAS sampler is successfully applied to the classical problem of Wiener system identification, and it is also used for inference in the challenging class of non-Markovian latent variable models.Many nonlinear models encountered in practice contain some tractable substructure. As a second problem considered in this thesis, we develop Monte Carlo methods capable of exploiting such substructures to obtain more accurate estimators than what is provided otherwise. For the filtering problem, this can be done by using the well known Rao-Blackwellized particle filter (RBPF). The RBPF is analysed in terms of asymptotic variance, resulting in an expression for the performance gain offered by Rao-Blackwellization. Furthermore, a Rao-Blackwellized particle smoother is derived, capable of addressing the smoothing problem in so called mixed linear/nonlinear state-space models. The idea of Rao-Blackwellization is also used to develop an online algorithm for Bayesian parameter inference in nonlinear state-space models with affine parameter dependencies.

    List of papers
    1. Backward simulation methods for Monte Carlo statistical inference
    Open this publication in new window or tab >>Backward simulation methods for Monte Carlo statistical inference
    2013 (English)In: Foundations and Trends in Machine Learning, ISSN 1935-8237, Vol. 6, no 1, p. 1-143Article in journal (Refereed) Published
    Abstract [en]

    Monte Carlo methods, in particular those based on Markov chains and on interacting particle systems, are by now tools that are routinely used in machine learning. These methods have had a profound impact on statistical inference in a wide range of application areas where probabilistic models are used. Moreover, there are many algorithms in machine learning which are based on the idea of processing the data sequentially, first in the forward direction and then in the backward direction. In this tutorial we will review a branch of Monte Carlo methods based on the forward-backward idea, referred to as backward simulators. These methods are useful for learning and inference in probabilistic models containing latent stochastic processes. The theory and practice of backward simulation algorithms have undergone a significant development in recent years and the algorithms keep finding new applications. The foundation for these methods is sequential Monte Carlo (SMC). SMC-based backward simulators are capable of addressing smoothing problems in sequential latent variable models, such as general, nonlinear/non-Gaussian state-space models (SSMs). However, we will also clearly show that the underlying backward simulation idea is by no means restricted to SSMs. Furthermore, backward simulation plays an important role in recent developments of Markov chain Monte Carlo (MCMC) methods. Particle MCMC is a systematic way of using SMC within MCMC. In this framework, backward simulation gives us a way to significantly improve the performance of the samplers. We review and discuss several related backward-simulation-based methods for state inference as well as learning of static parameters, both using a frequentistic and a Bayesian approach.

    Keywords
    Bayesian learning, Markov chain Monte Carlo, Nonlinear signal processing, Particle smoothing, Sequential Monte Carlo
    National Category
    Control Engineering Probability Theory and Statistics
    Identifiers
    urn:nbn:se:liu:diva-98294 (URN)10.1561/2200000045 (DOI)
    Projects
    CNDMCADICS
    Funder
    Swedish Research Council
    Available from: 2013-10-07 Created: 2013-10-07 Last updated: 2013-10-08
    2. Ancestor Sampling for Particle Gibbs
    Open this publication in new window or tab >>Ancestor Sampling for Particle Gibbs
    2012 (English)In: Proceedings of the 26th Conference on Neural Information Processing Systems, 2012Conference paper, Published paper (Refereed)
    Abstract [en]

    We present a novel method in the family of particle MCMC methods that we refer to as particle Gibbs with ancestor sampling (PG-AS). Similarly to the existing PG with backward simulation (PG-BS) procedure, we use backward sampling to (considerably) improve the mixing of the PG kernel. Instead of using separate forward and backward sweeps as in PG-BS, however, we achieve the same effect in a single forward sweep. We apply the PG-AS framework to the challenging class of non-Markovian state-space models. We develop a truncation strategy of these models that is applicable in principle to any backward-simulation-based method, but which is particularly well suited to the PG-AS framework. In particular, as we show in a simulation study, PG-AS can yield an order-of-magnitude improved accuracy relative to PG-BS due to its robustness to the truncation error. Several application examples are discussed, including Rao-Blackwellized particle smoothing and inference in degenerate state-space models.

    Keywords
    Particle Gibbs, Sampling
    National Category
    Probability Theory and Statistics Control Engineering
    Identifiers
    urn:nbn:se:liu:diva-88610 (URN)9781627480031 (ISBN)
    Conference
    26th Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 3-6 December, 2012
    Projects
    CADICSCNDM
    Funder
    Swedish Research Council
    Available from: 2013-02-13 Created: 2013-02-13 Last updated: 2013-10-08
    3. Bayesian semiparametric Wiener system identification
    Open this publication in new window or tab >>Bayesian semiparametric Wiener system identification
    2013 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 49, no 7, p. 2053-2063Article in journal (Refereed) Published
    Abstract [en]

    We present a novel method for Wiener system identification. The method relies on a semiparametric, i.e. a mixed parametric/nonparametric, model of a Wiener system. We use a state-space model for the linear dynamical system and a nonparametric Gaussian process model for the static nonlinearity. We avoid making strong assumptions, such as monotonicity, on the nonlinear mapping. Stochastic disturbances, entering both as measurement noise and as process noise, are handled in a systematic manner. The nonparametric nature of the Gaussian process allows us to handle a wide range of nonlinearities without making problem-specific parameterizations. We also consider sparsity-promoting priors, based on generalized hyperbolic distributions, to automatically infer the order of the underlying dynamical system. We derive an inference algorithm based on an efficient particle Markov chain Monte Carlo method, referred to as particle Gibbs with ancestor sampling. The method is profiled on two challenging identification problems with good results. Blind Wiener system identification is handled as a special case.

    Place, publisher, year, edition, pages
    Elsevier, 2013
    Keywords
    System identification, Wiener, Block-oriented models, Gaussian process, Semiparametric, Particle filter, Ancestor sampling, Particle Markov chain Monte Carlo
    National Category
    Engineering and Technology
    Identifiers
    urn:nbn:se:liu:diva-95954 (URN)10.1016/j.automatica.2013.03.021 (DOI)000321233900011 ()
    Note

    Funding Agencies|project Calibrating Nonlinear Dynamical Models|621-2010-5876|Swedish Research Council||CADICS||Linnaeus Center||

    Available from: 2013-08-19 Created: 2013-08-12 Last updated: 2017-12-06
    4. An Efficient Stochastic Approximation EM Algorithm using Conditional Particle Filters
    Open this publication in new window or tab >>An Efficient Stochastic Approximation EM Algorithm using Conditional Particle Filters
    2013 (English)In: Proceedings of the 38th International Conference on Acoustics, Speech, and Signal Processing, IEEE conference proceedings, 2013, p. 6274-6278Conference paper, Published paper (Refereed)
    Abstract [en]

    I present a novel method for maximum likelihood parameter estimation in nonlinear/non-Gaussian state-space models. It is an expectation maximization (EM) like method, which uses sequential Monte Carlo (SMC) for the intermediate state inference problem. Contrary to existing SMC-based EM algorithms, however, it makes efficient use of the simulated particles through the use of particle Markov chain Monte Carlo (PMCMC) theory. More precisely, the proposed method combines the efficient conditional particle filter with ancestor sampling (CPF-AS) with the stochastic approximation EM (SAEM) algorithm. This results in a procedure which does not rely on asymptotics in the number of particles for convergence, meaning that the method is very computationally competitive. Indeed, the method is evaluated in a simulation study, using a small number of particles with promising results.

    Place, publisher, year, edition, pages
    IEEE conference proceedings, 2013
    Keywords
    Maximum likelihood, Stochastic approximation, Conditional particle filter
    National Category
    Control Engineering Signal Processing
    Identifiers
    urn:nbn:se:liu:diva-93459 (URN)10.1109/ICASSP.2013.6638872 (DOI)000329611506087 ()
    Conference
    38th International Conference on Acoustics, Speech, and Signal Processing, Vancouver, Canada, 26-31 May, 2013
    Projects
    CNDMCADICS
    Funder
    Swedish Research Council, 621-2010-5876
    Available from: 2013-06-03 Created: 2013-06-03 Last updated: 2014-02-20
    5. Rao-Blackwellized Particle Smoothers for Mixed Linear/Nonlinear State-Space Models
    Open this publication in new window or tab >>Rao-Blackwellized Particle Smoothers for Mixed Linear/Nonlinear State-Space Models
    2013 (English)In: Proceedings of the 38th International Conference on Acoustics, Speech, and Signal Processing, IEEE conference proceedings, 2013, p. 6288-6292Conference paper, Published paper (Refereed)
    Abstract [en]

    We consider the smoothing problem for a class of conditionally linear Gaussian state-space (CLGSS) models, referred to as mixed linear/nonlinear models. In contrast to the better studied hierarchical CLGSS models, these allow for an intricate cross dependence between the linear and the nonlinear parts of the state vector. We derive a Rao-Blackwellized particle smoother (RBPS) for this model class by exploiting its tractable substructure. The smoother is of the forward filtering/backward simulation type. A key feature of the proposed method is that, unlike existing RBPS for this model class, the linear part of the state vector is marginalized out in both the forward direction and in the backward direction.

    Place, publisher, year, edition, pages
    IEEE conference proceedings, 2013
    Keywords
    Rao-Blackwellization, Particle smoothing, Backward simulation, Sequential Monte Carlo
    National Category
    Signal Processing Control Engineering
    Identifiers
    urn:nbn:se:liu:diva-93460 (URN)10.1109/ICASSP.2013.6638875 (DOI)000329611506090 ()
    Conference
    38th International Conference on Acoustics, Speech, and Signal Processing, Vancouver, Canada, 26-31 May, 2013
    Projects
    CNDMCADICS
    Funder
    Swedish Research Council, 621-2010-5876
    Available from: 2013-06-03 Created: 2013-06-03 Last updated: 2014-02-20Bibliographically approved
    6. A non-degenerate Rao-Blackwellised particle filter for estimating static parameters in dynamical models
    Open this publication in new window or tab >>A non-degenerate Rao-Blackwellised particle filter for estimating static parameters in dynamical models
    2012 (English)In: Proceedings of the 16th IFAC Symposium on System Identification, 2012Conference paper, Oral presentation only (Refereed)
    Abstract [en]

    The particle filter (PF) has emerged as a powerful tool for solving nonlinear and/or non-Gaussian filtering problems. When some of the states enter the model linearly, this can be exploited by using particles only for the "nonlinear" states and employing conditional Kalman filters for the "linear" states; this leads to the Rao-Blackwellised particle filter (RBPF). However, it is well known that the PF fails when the state of the model contains some static parameter. This is true also for the RBPF, even if the static states are marginalised analytically by a Kalman filter. The reason is that the posterior density of the static states is computed conditioned on the nonlinear particle trajectories, which are bound to degenerate over time. To circumvent this problem, we propose a method for targeting the posterior parameter density, conditioned on just the current nonlinear state. This results in an RBPF-like method, capable of recursive identification of nonlinear dynamical models with affine parameter dependencies.

    Keywords
    Particle Filtering/Monte Carlo Methods; Nonlinear System Identification; Recursive Identification
    National Category
    Electrical Engineering, Electronic Engineering, Information Engineering
    Identifiers
    urn:nbn:se:liu:diva-81271 (URN)10.3182/20120711-3-BE-2027.00184 (DOI)
    Conference
    16th IFAC Symposium on System Identification, Brussels, Belgium, July 11-13, 2012.
    Projects
    CADICSCNDM
    Funder
    Swedish Research Council
    Available from: 2012-09-10 Created: 2012-09-10 Last updated: 2013-10-08
    7. An Explicit Variance Reduction Expression for the Rao-Blackwellised Particle Filter
    Open this publication in new window or tab >>An Explicit Variance Reduction Expression for the Rao-Blackwellised Particle Filter
    2011 (English)In: Proceedings of the 18th IFAC World Congress, 2011, p. 11979-11984Conference paper, Published paper (Refereed)
    Abstract [en]

    Particle filters (PFs) have shown to be very potent tools for state estimation in nonlinear and/or non-Gaussian state-space models. For certain models, containing a conditionally tractable substructure (typically conditionally linear Gaussian or with finite support), it is possible to exploit this structure in order to obtain more accurate estimates. This has become known as Rao-Blackwellised particle filtering (RBPF). However, since the RBPF is typically more computationally demanding than the standard PF per particle, it is not always beneficial to resort to Rao-Blackwellisation. For the same computational effort, a standard PF with an increased number of particles, which would also increase the accuracy, could be used instead. In this paper, we have analysed the asymptotic variance of the RBPF and provide an explicit expression for the obtained variance reduction. This expression could be used to make an efficient discrimination of when to apply Rao-Blackwellisation, and when not to.

    Keywords
    Particle filtering, Monte-Carlo methods, Rao-Blackwellised particle filter, Marginalised particle filter, Rao-Blackwellisation, Variance reduction
    National Category
    Control Engineering
    Identifiers
    urn:nbn:se:liu:diva-81259 (URN)10.3182/20110828-6-IT-1002.02920 (DOI)978-3-902661-93-7 (ISBN)
    Conference
    18th IFAC World Congress, Milano, Italy 28 August-2 September, 2011
    Projects
    CADICSCNDM
    Funder
    Swedish Research Council
    Available from: 2012-09-10 Created: 2012-09-10 Last updated: 2013-10-08
    Download full text (pdf)
    Particle filters and Markov chains for learning of dynamical systems
    Download (pdf)
    omslag
  • 22. Order onlineBuy this publication >>
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Rao-Blackwellised particle methods for inference and identification2011Licentiate thesis, monograph (Other academic)
    Abstract [en]

    We consider the two related problems of state inference in nonlinear dynamical systems and nonlinear system identification. More precisely, based on noisy observations from some (in general) nonlinear and/or non-Gaussian dynamical system, we seek to estimate the system state as well as possible unknown static parameters of the system. We consider two different aspects of the state inference problem, filtering and smoothing, with the emphasis on the latter. To address the filtering and smoothing problems, we employ sequential Monte Carlo (SMC) methods, commonly referred to as particle filters (PF) and particle smoothers (PS).

    Many nonlinear models encountered in practice contain some tractable substructure. If this is the case, a natural idea is to try to exploit this substructure to obtain more accurate estimates than what is provided by a standard particle method. For the filtering problem, this can be done by using the well-known Rao-Blackwellised particle filter (RBPF). In this thesis, we analyse the RBPF and provide explicit expressions for the variance reduction that is obtained from Rao-Blackwellisation. Furthermore, we address the smoothing problem and develop a novel Rao-Blackwellised particle smoother (RBPS), designed to exploit a certain tractable substructure in the model.

    Based on the RBPF and the RBPS we propose two different methods for nonlinear system identification. The first is a recursive method referred to as the Rao-Blackwellised marginal particle filter (RBMPF). By augmenting the state variable with the unknown parameters, a nonlinear filter can be applied to address the parameter estimation problem. However, if the model under study has poor mixing properties, which is the case if the state variable contains some static parameter, SMC filters such as the PF and the RBPF are known to degenerate. To circumvent this we introduce a so called “mixing” stage in the RBMPF, which makes it more suitable for models with poor mixing properties.

    The second identification method is referred to as RBPS-EM and is designed for maximum likelihood parameter estimation in a type of mixed linear/nonlinear Gaussian statespace models. The method combines the expectation maximisation (EM) algorithm with the RBPS mentioned above, resulting in an identification method designed to exploit the tractable substructure present in the model.

    Download full text (pdf)
    Rao-Blackwellised particle methods for inference and identification
    Download (pdf)
    COVER01
  • 23.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Bunch, Pete
    University of Cambridge, United Kingdom.
    Godsill, Simon J.
    University of Cambridge, United Kingdom.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Rao-Blackwellized Particle Smoothers for Mixed Linear/Nonlinear State-Space Models2013In: Proceedings of the 38th International Conference on Acoustics, Speech, and Signal Processing, IEEE conference proceedings, 2013, p. 6288-6292Conference paper (Refereed)
    Abstract [en]

    We consider the smoothing problem for a class of conditionally linear Gaussian state-space (CLGSS) models, referred to as mixed linear/nonlinear models. In contrast to the better studied hierarchical CLGSS models, these allow for an intricate cross dependence between the linear and the nonlinear parts of the state vector. We derive a Rao-Blackwellized particle smoother (RBPS) for this model class by exploiting its tractable substructure. The smoother is of the forward filtering/backward simulation type. A key feature of the proposed method is that, unlike existing RBPS for this model class, the linear part of the state vector is marginalized out in both the forward direction and in the backward direction.

    Download full text (pdf)
    fulltext
  • 24.
    Lindsten, Fredrik
    et al.
    Uppsala universitet, Avdelningen för systemteknik, Sweden.
    Bunch, Pete
    Department of Engineering, University of Cambridge, Cambridge, UK.
    Särkkä, Simo
    Department of Electrical Engineering and Automation, Aalto University, Aalto, Finland.
    Schön, Thomas B.
    Uppsala universitet, Avdelningen för systemteknik, Sweden.
    Godsill, Simon J.
    Department of Engineering, University of Cambridge, Cambridge, UK.
    Rao–Blackwellized particle smoothers for conditionally linear Gaussian models2016In: IEEE Journal on Selected Topics in Signal Processing, ISSN 1932-4553, E-ISSN 1941-0484, Vol. 10, no 2, p. 353-365Article in journal (Refereed)
    Abstract [en]

    Sequential Monte Carlo (SMC) methods, such as the particle filter, are by now one of the standard computational techniques for addressing the filtering problem in general state-space models. However, many applications require post-processing of data offline. In such scenarios the smoothing problem-in which all the available data is used to compute state estimates-is of central interest. We consider the smoothing problem for a class of conditionally linear Gaussian models. We present a forward-backward-type Rao-Blackwellized particle smoother (RBPS) that is able to exploit the tractable substructure present in these models. Akin to the well known Rao-Blackwellized particle filter, the proposed RBPS marginalizes out a conditionally tractable subset of state variables, effectively making use of SMC only for the “intractable part” of the model. Compared to existing RBPS, two key features of the proposed method are: 1) it does not require structural approximations of the model, and 2) the aforementioned marginalization is done both in the forward direction and in the backward direction.

  • 25.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Callmer, Jonas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Ohlsson, Henrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Törnqvist, David
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Gustafsson, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Geo-Referencing for UAV Navigation using Environmental Classification2010Report (Other academic)
    Abstract [en]

    A UAV navigation system relying on GPS is vulnerable to signal failure, making a drift free backup system necessary. We introduce a vision based geo-referencing system that uses pre-existing maps to reduce the long term drift. The system classifies an image according to its environmental content and thereafter matches it to an environmentally classified map over the operational area. This map matching provides a measurement of the absolute location of the UAV, that can easily be incorporated into a sensor fusion framework. Experiments show that the geo-referencing system reduces the long term drift in UAV navigation, enhancing the ability of the UAV to navigate accurately over large areas without the use of GPS.

    Download full text (pdf)
    FULLTEXT01
  • 26.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Callmer, Jonas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Ohlsson, Henrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Törnqvist, David
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Gustafsson, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Geo-Referencing for UAV Navigation using Environmental Classification2010In: Proceedings of the 2010 IEEE International Conference on Robotics and Automation, 2010, p. 1420-1425Conference paper (Refereed)
    Abstract [en]

    A UAV navigation system relying on GPS is vulnerable to signal failure, making a drift free backup system necessary. We introduce a vision based geo-referencing system that uses pre-existing maps to reduce the long term drift. The system classifies an image according to its environmental content and thereafter matches it to an environmentally classified map over the operational area. This map matching provides a measurement of the absolute location of the UAV, that can easily be incorporated into a sensor fusion framework. Experiments show that the geo-referencing system reduces the long term drift in UAV navigation, enhancing the ability of the UAV to navigate accurately over large areas without the use of GPS.

  • 27.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Faculty of Science & Engineering. University of Cambridge, England.
    Douc, Randal
    Telecom SudParis, France.
    Moulines, Eric
    Telecom ParisTech, France.
    Uniform Ergodicity of the Particle Gibbs Sampler2015In: Scandinavian Journal of Statistics, ISSN 0303-6898, E-ISSN 1467-9469, Vol. 42, no 3, p. 775-797Article in journal (Refereed)
    Abstract [en]

    The particle Gibbs sampler is a systematic way of using a particle filter within Markov chain Monte Carlo. This results in an off-the-shelf Markov kernel on the space of state trajectories, which can be used to simulate from the full joint smoothing distribution for a state space model in a Markov chain Monte Carlo scheme. We show that the particle Gibbs Markov kernel is uniformly ergodic under rather general assumptions, which we will carefully review and discuss. In particular, we provide an explicit rate of convergence, which reveals that (i) for fixed number of data points, the convergence rate can be made arbitrarily good by increasing the number of particles and (ii) under general mixing assumptions, the convergence rate can be kept constant by increasing the number of particles superlinearly with the number of observations. We illustrate the applicability of our result by studying in detail a common stochastic volatility model with a non-compact state space.

    Download full text (pdf)
    fulltext
  • 28.
    Lindsten, Fredrik
    et al.
    Uppsala University, Sweden.
    Johansen, A. M.
    University of Warwick, England.
    Andersson Naesseth, Christian
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Faculty of Science & Engineering.
    Kirkpatrick, B.
    Intrepid Net Comp, MT USA.
    Schön, T. B.
    Uppsala University, Sweden.
    Aston, J. A. D.
    University of Cambridge, England.
    Bouchard-Cote, A.
    University of British Columbia, Canada.
    Divide-and-Conquer With Sequential Monte Carlo2017In: Journal of Computational And Graphical Statistics, ISSN 1061-8600, E-ISSN 1537-2715, Vol. 26, no 2, p. 445-458Article in journal (Refereed)
    Abstract [en]

    We propose a novel class of Sequential Monte Carlo (SMC) algorithms, appropriate for inference in probabilistic graphical models. This class of algorithms adopts a divide-and-conquer approach based upon an auxiliary tree-structured decomposition of the model of interest, turning the overall inferential task into a collection of recursively solved subproblems. The proposed method is applicable to a broad class of probabilistic graphical models, including models with loops. Unlike a standard SMC sampler, the proposed divide-and-conquer SMC employs multiple independent populations of weighted particles, which are resampled, merged, and propagated as the method progresses. We illustrate empirically that this approach can outperform standard methods in terms of the accuracy of the posterior expectation and marginal likelihood approximations. Divide-and-conquer SMC also opens up novel parallel implementation options and the possibility of concentrating the computational effort on the most challenging subproblems. We demonstrate its performance on a Markov random field and on a hierarchical logistic regression problem. Supplementary materials including proofs and additional numerical results are available online.

  • 29.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Jordan, Michael I.
    University of Calif Berkeley, CA 94720 USA; University of Calif Berkeley, CA 94720 USA.
    Schon, Thomas B.
    Uppsala University, Sweden.
    Particle Gibbs with Ancestor Sampling2014In: Journal of machine learning research, ISSN 1532-4435, E-ISSN 1533-7928, Vol. 15, p. 2145-2184Article in journal (Refereed)
    Abstract [en]

    Particle Markov chain Monte Carlo (PMCMC) is a systematic way of combining the two main tools used for Monte Carlo statistical inference: sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC). We present a new PMCMC algorithm that we refer to as particle Gibbs with ancestor sampling (PGAS). PGAS provides the data analyst with an off-the-shelf class of Markov kernels that can be used to simulate, for instance, the typically high-dimensional and highly autocorrelated state trajectory in a state-space model. The ancestor sampling procedure enables fast mixing of the PGAS kernel even when using seemingly few particles in the underlying SMC sampler. This is important as it can significantly reduce the computational burden that is typically associated with using SMC. PGAS is conceptually similar to the existing PG with backward simulation (PGBS) procedure. Instead of using separate forward and backward sweeps as in PGBS, however, we achieve the same effect in a single forward sweep. This makes PGAS well suited for addressing inference problems not only in state-space models, but also in models with more complex dependencies, such as non-Markovian, Bayesian nonparametric, and general probabilistic graphical models.

    Download full text (pdf)
    fulltext
  • 30.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Jordan, Michael I.
    University of California, Berkeley.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Ancestor Sampling for Particle Gibbs2012In: Proceedings of the 26th Conference on Neural Information Processing Systems, 2012Conference paper (Refereed)
    Abstract [en]

    We present a novel method in the family of particle MCMC methods that we refer to as particle Gibbs with ancestor sampling (PG-AS). Similarly to the existing PG with backward simulation (PG-BS) procedure, we use backward sampling to (considerably) improve the mixing of the PG kernel. Instead of using separate forward and backward sweeps as in PG-BS, however, we achieve the same effect in a single forward sweep. We apply the PG-AS framework to the challenging class of non-Markovian state-space models. We develop a truncation strategy of these models that is applicable in principle to any backward-simulation-based method, but which is particularly well suited to the PG-AS framework. In particular, as we show in a simulation study, PG-AS can yield an order-of-magnitude improved accuracy relative to PG-BS due to its robustness to the truncation error. Several application examples are discussed, including Rao-Blackwellized particle smoothing and inference in degenerate state-space models.

  • 31.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Nordlund, Per-Johan
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Gustafsson, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Conflict Detection Metrics for Aircraft Sense and Avoid Systems2009Report (Other academic)
    Abstract [en]

    The task of an airborne collision avoidance system is to continuously evaluate the risk of collision and in the case of too high risk initiate an evasive action. The traditional way to assess risk is to focus on a critical point of time. A recently proposed alternative is to evaluate the cumulated risk over time. It is the purpose of this contribution to evaluate the difference between these two concepts and also to validate an approximate method for computing the cumulated risk, suitable for real-time implementations. For this purpose, random scenarios are generated from stochastic models created from observed conflicts. A realistic tracking filter, based on angle-only measurements, is used to produce uncertain state estimates which are used for risk assessment. It is shown that the cumulated risk is much more robust to estimation accuracy than the maximum of the instantaneous risk. The intended application is for unmanned aerial vehicles to be used in civilian airspace, but a real mid-air collision scenario between two traffic aircraft is studied as well.

    Download full text (pdf)
    FULLTEXT01
  • 32.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Nordlund, Per-Johan
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Gustafsson, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Conflict Detection Metrics for Aircraft Sense and Avoid Systems2009In: Proceedings of the 7th IFAC Symposium on Fault Detection, Supervision and Safety of Technical Processes, 2009, p. 65-70Conference paper (Refereed)
    Abstract [en]

    The task of an airborne collision avoidance system is to continuously evaluate the risk of collision and in the case of too high risk initiate an evasive action. The traditional way to assess risk is to focus on a critical point of time. A recently proposed alternative is to evaluate the cumulated risk over time. It is the purpose of this contribution to evaluate the difference between these two concepts and also to validate an approximate method for computing the cumulated risk, suitable for real-time implementations. For this purpose, random scenarios are generated from stochastic models created from observed conflicts. A realistic tracking filter, based on angle-only measurements, is used to produce uncertain state estimates which are used for risk assessment. It is shown that the cumulated risk is much more robust to estimation accuracy than the maximum of the instantaneous risk. The intended application is for unmanned aerial vehicles to be used in civilian airspace, but a real mid-air collision scenario between two traffic aircraft is studied as well.

    Download full text (pdf)
    FULLTEXT01
  • 33.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Ohlsson, Henrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Ljung, Lennart
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Clustering using Sum-of-Norms Regularization: With Application to Particle Filter Output Computation2011In: Proceedings of the 2011 IEEE Statistical Signal Processing Workshop, 2011, p. 201-204Conference paper (Refereed)
    Abstract [en]

    We present a novel clustering method, formulated as a convex optimization problem. The method is based on over-parameterization and uses a sum-of-norms (SON) regularization to control the trade-off between the model fit and the number of clusters. Hence, the number of clusters can be automatically adapted to best describe the data, and need not to be specified a priori. We apply SON clustering to cluster the particles in a particle filter, an application where the number of clusters is often unknown and time varying, making SON clustering an attractive alternative.

  • 34.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Ohlsson, Henrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Ljung, Lennart
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Clustering using Sum-of-Norms Regularization: With Application to Particle Filter Output Computation2011Report (Other academic)
    Abstract [en]

    We present a novel clustering method, formulated as a convex optimization problem. The method is based on over-parameterization and uses a sum-of-norms (SON) regularization to control the trade-off between the model fit and the number of clusters. Hence, the number of clusters can be automatically adapted to best describe the data, and need not to be specified a priori. We apply SON clustering to cluster the particles in a particle filter, an application where the number of clusters is often unknown and time varying, making SON clustering an attractive alternative.

    Download full text (pdf)
    FULLTEXT01
  • 35.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Ohlsson, Henrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Ljung, Lennart
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Just Relax and Come Clustering!: A Convexification of k-Means Clustering2011Report (Other academic)
    Abstract [en]

    k-means clustering is a popular approach to clustering. It is easy to implement and intuitive but has the disadvantage of being sensitive to initialization due to an underlying nonconvex optimization problem. In this paper, we derive an equivalent formulation of k-means clustering. The formulation takes the form of a L0-regularized least squares problem. We then propose a novel convex, relaxed, formulation of k-means clustering. The sum-of-norms regularized least squares formulation inherits many desired properties of k-means but has the advantage of being independent of initialization.

    Download full text (pdf)
    FULLTEXT01
  • 36.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Identification of Mixed Linear/Nonlinear State-Space Models2010In: Proceedings of the 49th IEEE Conference on Decision and Control, 2010, p. 6377-6382Conference paper (Refereed)
    Abstract [en]

    The primary contribution of this paper is an algorithm capable of identifying parameters in certain mixed linear/nonlinear state-space models, containing conditionally linear Gaussian substructures. More specifically, we employ the standard maximum likelihood framework and derive an expectation maximization type algorithm. This involves a nonlinear smoothing problem for the state variables, which for the conditionally linear Gaussian system can be efficiently solved using a so called Rao-Blackwellized particle smoother (RBPS). As a secondary contribution of this paper we extend an existing RBPS to be able to handle the fully interconnected model under study.

     

    Download full text (pdf)
    FULLTEXT02
  • 37.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Inference in Mixed Linear/Nonlinear State-Space Models using Sequential Monte Carlo2010Report (Other academic)
    Abstract [en]

    In this work we apply sequential Monte Carlo methods, i.e., particle filters and smoothers, to estimate the state in a certain class of mixed linear/nonlinear state-space models. Such a model has an inherent conditionally linear Gaussian substructure. By utilizing this structure we are able to address even high-dimensional nonlinear systems using Monte Carlo methods, as long as only a few of the states enter nonlinearly. First, we consider the filtering problem and give a self-constained derivation of the well known Rao-Blackellized particle filter. Therafter we turn to the smoothing problem and derive a Rao-Blackwellized particle smoother capable of handling the fully interconnected model under study.

    Download full text (pdf)
    FULLTEXT01
  • 38.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    On the Use of Backward Simulation in the Particle Gibbs Sampler2012In: Proceedings of the 37th IEEE International Conference on Acoustics, Speech, and Signal Processing, IEEE , 2012, p. 3845-3848Conference paper (Refereed)
    Abstract [en]

    The particle Gibbs (PG) sampler was introduced in [Andrieu et al. (2010)] as a way to incorporate a particle filter (PF) in a Markov chain Monte Carlo (MCMC) sampler. The resulting method was shown to be an efficient tool for joint Bayesian parameter and state inference in nonlinear, non-Gaussian state-space models. However, the mixing of the PG kernel can be very poor when there is severe degeneracy in the PF. Hence, the success of the PG sampler heavily relies on the, often unrealistic, assumption that we can implement a PF without suffering from any considerate degeneracy. However, as pointed out by Whiteley in the discussion following [Andrieu et al. (2010)], the mixing can be improved by adding a backward simulation step to the PG sampler. Here, we investigate this further, derive an explicit PG sampler with backward simulation (denoted PG-BSi) and show that this indeed is a valid MCMC method. Furthermore, we show in a numerical example that backward simulation can lead to a considerable increase in performance over the standard PG sampler.

  • 39.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas B.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Backward simulation methods for Monte Carlo statistical inference2013In: Foundations and Trends in Machine Learning, ISSN 1935-8237, Vol. 6, no 1, p. 1-143Article in journal (Refereed)
    Abstract [en]

    Monte Carlo methods, in particular those based on Markov chains and on interacting particle systems, are by now tools that are routinely used in machine learning. These methods have had a profound impact on statistical inference in a wide range of application areas where probabilistic models are used. Moreover, there are many algorithms in machine learning which are based on the idea of processing the data sequentially, first in the forward direction and then in the backward direction. In this tutorial we will review a branch of Monte Carlo methods based on the forward-backward idea, referred to as backward simulators. These methods are useful for learning and inference in probabilistic models containing latent stochastic processes. The theory and practice of backward simulation algorithms have undergone a significant development in recent years and the algorithms keep finding new applications. The foundation for these methods is sequential Monte Carlo (SMC). SMC-based backward simulators are capable of addressing smoothing problems in sequential latent variable models, such as general, nonlinear/non-Gaussian state-space models (SSMs). However, we will also clearly show that the underlying backward simulation idea is by no means restricted to SSMs. Furthermore, backward simulation plays an important role in recent developments of Markov chain Monte Carlo (MCMC) methods. Particle MCMC is a systematic way of using SMC within MCMC. In this framework, backward simulation gives us a way to significantly improve the performance of the samplers. We review and discuss several related backward-simulation-based methods for state inference as well as learning of static parameters, both using a frequentistic and a Bayesian approach.

  • 40.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas B.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Maximum Likelihood Estimation in Mixed Linear/Nonlinear State-Space Models2010Report (Other academic)
    Abstract [en]

    The primary contribution of this paper is an algorithm capable of identifying parameters in certain mixed linear/nonlinear state-space models, containing conditionally linear Gaussian substructures. More specifically, we employ the standard maximum likelihood framework and derive an expectation maximization type algorithm. This involves a nonlinear smoothing problem for the state variables, which for the conditionally linear Gaussian system can be efficiently solved using so called Rao-Blackwellized particle smoother (RBPS). As a secondary contribution of this paper we extend an existing RBPS to be able to handle the fully interconnected model under study.

    Download full text (pdf)
    FULLTEXT01
  • 41.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas B.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Rao-Blackwellized Particle Smoothers for Mixed Linear/Nonlinear State-Space Models2011Report (Other academic)
    Abstract [en]

    We consider the smoothing problem for a class of conditionally linear Gaussian state-space (CLGSS) models, referred to as mixed linear/nonlinear models. In contrast to the better studied hierarchical CLGSS models, these allow for an intricate cross dependence between the linear and the nonlinear parts of the state vector. We derive a Rao-Blackwellized particle smoother (RBPS) for this model class by exploiting its tractable substructure. The smoother is of the forward filtering/backward simulation type. A key feature of the proposed method is that, unlike existing RBPS for this model class, the linear part of the state vector is marginalized out in both the forward direction and in the backward direction.

    Download full text (pdf)
    FULLTEXT01
  • 42.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas B.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Jordan, Michael I.
    University of California, Berkeley, USA.
    A Semiparametric Bayesian Approach to Wiener System Identification2012In: Proceedings of the 16th IFAC Symposium on System Identification, 2012, p. 1137-1142Conference paper (Refereed)
    Abstract [en]

    We consider a semiparametric, i.e. a mixed parametric/nonparametric, model of a Wiener system. We use a state-space model for the linear dynamical system and a nonparametric Gaussian process (GP) model for the static nonlinearity. The GP model is a flexible model that can describe different types of nonlinearities while avoiding making strong assumptions such as monotonicity. We derive an inferential method based on recent advances in Monte Carlo statistical methods, known as Particle Markov Chain Monte Carlo (PMCMC). The idea underlying PMCMC is to use a particle filter (PF) to generate a sample state trajectory in a Markov chain Monte Carlo sampler. We use a recently proposed PMCMC sampler, denoted particle Gibbs with backward simulation, which has been shown to be efficient even when we use very few particles in the PF. The resulting method is used in a simulation study to identify two different Wiener systems with non-invertible nonlinearities.

  • 43.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas B.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Olsson, Jimmy
    Lund University, Sweden.
    An Explicit Variance Reduction Expression for the Rao-Blackwellised Particle Filter2010Report (Other academic)
    Abstract [en]

    Particle filters (PFs) have shown to be very potent tools for state estimation in nonlinear and/or non-Gaussian state-space models. For certain models, containing a conditionally tractable substructure (typically conditionally linear Gaussian or with finite support), it is possible to exploit this structure in order to obtain more accurate estimates. This has become known as Rao-Blackwellised particle filtering (RBPF). However, since the RBPF is typically more computationally demanding than the standard PF per particle, it is not always beneficial to resort to Rao-Blackwellisation. For the same computational effort, a standard PF with an increased number of particles, which would also increase the accuracy, could be used instead. In this paper, we have analysed the asymptotic variance of the RBPF and provide an explicit expression for the obtained variance reduction. This expression could be used to make an efficient discrimination of when to apply Rao-Blackwellisation, and when not to.

    Download full text (pdf)
    FULLTEXT01
  • 44.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas B.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Svensson, Lennart
    Division of Signals and Systems, Chalmers University.
    A non-degenerate Rao-Blackwellised particle filter for estimating static parameters in dynamical models2012In: Proceedings of the 16th IFAC Symposium on System Identification, 2012Conference paper (Refereed)
    Abstract [en]

    The particle filter (PF) has emerged as a powerful tool for solving nonlinear and/or non-Gaussian filtering problems. When some of the states enter the model linearly, this can be exploited by using particles only for the "nonlinear" states and employing conditional Kalman filters for the "linear" states; this leads to the Rao-Blackwellised particle filter (RBPF). However, it is well known that the PF fails when the state of the model contains some static parameter. This is true also for the RBPF, even if the static states are marginalised analytically by a Kalman filter. The reason is that the posterior density of the static states is computed conditioned on the nonlinear particle trajectories, which are bound to degenerate over time. To circumvent this problem, we propose a method for targeting the posterior parameter density, conditioned on just the current nonlinear state. This results in an RBPF-like method, capable of recursive identification of nonlinear dynamical models with affine parameter dependencies.

  • 45.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Jordan, Michael I.
    University of California, Berkeley, USA.
    A Semiparametric Bayesian Approach to Wiener System Identification2011Report (Other academic)
    Abstract [en]

    We consider a semiparametric, i.e. a mixed parametric/nonparametric, model of a Wiener system. We use a state-space model for the linear dynamical system and a nonparametric Gaussian process (GP) model for the static nonlinearity. The GP model is a flexible model that can describe different types of nonlinearities while avoiding making strong assumptions such as monotonicity. We derive an inferential method based on recent advances in Monte Carlo statistical methods, known as Particle Markov Chain Monte Carlo (PMCMC). The idea underlying PMCMC is to use a particle filter (PF) to generate a sample state trajectory in a Markov chain Monte Carlo sampler. We use a recently proposed PMCMC sampler, denoted particle Gibbs with backward simulation, which has been shown to be efficient even when we use very few particles in the PF. The resulting method is used in a simulation study to identify two different Wiener systems with non-invertible nonlinearities.

    Download full text (pdf)
    FULLTEXT01
  • 46.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Jordan, Michael I.
    University of Calif Berkeley, CA USA .
    Bayesian semiparametric Wiener system identification2013In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 49, no 7, p. 2053-2063Article in journal (Refereed)
    Abstract [en]

    We present a novel method for Wiener system identification. The method relies on a semiparametric, i.e. a mixed parametric/nonparametric, model of a Wiener system. We use a state-space model for the linear dynamical system and a nonparametric Gaussian process model for the static nonlinearity. We avoid making strong assumptions, such as monotonicity, on the nonlinear mapping. Stochastic disturbances, entering both as measurement noise and as process noise, are handled in a systematic manner. The nonparametric nature of the Gaussian process allows us to handle a wide range of nonlinearities without making problem-specific parameterizations. We also consider sparsity-promoting priors, based on generalized hyperbolic distributions, to automatically infer the order of the underlying dynamical system. We derive an inference algorithm based on an efficient particle Markov chain Monte Carlo method, referred to as particle Gibbs with ancestor sampling. The method is profiled on two challenging identification problems with good results. Blind Wiener system identification is handled as a special case.

    Download full text (pdf)
    fulltext
  • 47.
    Lindsten, Fredrik
    et al.
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Schön, Thomas
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
    Olsson, Jimmy
    Lund University, Sweden.
    An Explicit Variance Reduction Expression for the Rao-Blackwellised Particle Filter2011In: Proceedings of the 18th IFAC World Congress, 2011, p. 11979-11984Conference paper (Refereed)
    Abstract [en]

    Particle filters (PFs) have shown to be very potent tools for state estimation in nonlinear and/or non-Gaussian state-space models. For certain models, containing a conditionally tractable substructure (typically conditionally linear Gaussian or with finite support), it is possible to exploit this structure in order to obtain more accurate estimates. This has become known as Rao-Blackwellised particle filtering (RBPF). However, since the RBPF is typically more computationally demanding than the standard PF per particle, it is not always beneficial to resort to Rao-Blackwellisation. For the same computational effort, a standard PF with an increased number of particles, which would also increase the accuracy, could be used instead. In this paper, we have analysed the asymptotic variance of the RBPF and provide an explicit expression for the obtained variance reduction. This expression could be used to make an efficient discrimination of when to apply Rao-Blackwellisation, and when not to.

  • 48.
    Naesseth, Christian A.
    et al.
    Columbia Univ, USA.
    Lindsten, Fredrik
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Science & Engineering.
    Schon, Thomas B.
    Uppsala Univ, Sweden.
    Elements of Sequential Monte Carlo2019In: FOUNDATIONS AND TRENDS IN MACHINE LEARNING, ISSN 1935-8237, Vol. 12, no 3, p. 187-306Article in journal (Refereed)
    Abstract [en]

    A core problem in statistics and probabilistic machine learning is to compute probability distributions and expectations. This is the fundamental problem of Bayesian statistics and machine learning, which frames all inference as expectations with respect to the posterior distribution. The key challenge is to approximate these intractable expectations. In this tutorial, we review sequential Monte Carlo (SMC), a random-sampling-based class of methods for approximate inference. First, we explain the basics of SMC, discuss practical issues, and review theoretical results. We then examine two of the main user design choices: the proposal distributions and the so called intermediate target distributions. We review recent results on how variational inference and amortization can be used to learn efficient proposals and target distributions. Next, we discuss the SMC estimate of the normalizing constant, how this can be used for pseudo-marginal inference and inference evaluation. Throughout the tutorial we illustrate the use of SMC on various models commonly used in machine learning, such as stochastic recurrent neural networks, probabilistic graphical models, and probabilistic programs.

    Download full text (pdf)
    fulltext
  • 49.
    Olmin, Amanda
    et al.
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Arts and Sciences.
    Lindqvist, Jakob
    Chalmers University of Technology, Gothenburg, Sweden.
    Svensson, Lennart
    Chalmers University of Technology, Gothenburg, Sweden.
    Lindsten, Fredrik
    Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Science & Engineering.
    Active Learning with Weak Supervision for Gaussian Processes2023In: Neural Information Processing 29th International Conference, ICONIP 2022, Virtual Event, November 22–26, 2022, Proceedings, Part V / [ed] M. Tanveer et al., Singapore: Springer Nature, 2023, p. 195-204Conference paper (Refereed)
    Abstract [en]

    Annotating data for supervised learning can be costly. When the annotation budget is limited, active learning can be used to select and annotate those observations that are likely to give the most gain in model performance. We propose an active learning algorithm that, in addition to selecting which observation to annotate, selects the precision of the annotation that is acquired. Assuming that annotations with low precision are cheaper to obtain, this allows the model to explore a larger part of the input space, with the same annotation budget. We build our acquisition function on the previously proposed BALD objective for Gaussian Processes, and empirically demonstrate the gains of being able to adjust the annotation precision in the active learning loop.

    The full text will be freely available from 2024-04-14 00:01
  • 50.
    Olmin, Amanda
    et al.
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Arts and Sciences.
    Lindsten, Fredrik
    Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning. Linköping University, Faculty of Science & Engineering.
    Robustness and Reliability When Training With Noisy Labels2022In: Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS) 2022, JMLR , 2022, Vol. 151, p. 922-942Conference paper (Refereed)
    Abstract [en]

    Labelling of data for supervised learning canbe costly and time-consuming and the riskof incorporating label noise in large data setsis imminent. When training a flexible discriminative model using a strictly proper loss,such noise will inevitably shift the solution towards the conditional distribution over noisylabels. Nevertheless, while deep neural networks have proven capable of fitting randomlabels, regularisation and the use of robustloss functions empirically mitigate the effectsof label noise. However, such observationsconcern robustness in accuracy, which is insufficient if reliable uncertainty quantificationis critical. We demonstrate this by analysingthe properties of the conditional distributionover noisy labels for an input-dependent noisemodel. In addition, we evaluate the set ofrobust loss functions characterised by noiseinsensitive, asymptotic risk minimisers. Wefind that strictly proper and robust loss functions both offer asymptotic robustness in accuracy, but neither guarantee that the finalmodel is calibrated. Moreover, even with robust loss functions, overfitting is an issue inpractice. With these results, we aim to explain observed robustness of common training practices, such as early stopping, to labelnoise. In addition, we aim to encourage thedevelopment of new noise-robust algorithmsthat not only preserve accuracy but that alsoensure reliability. 

    Download full text (pdf)
    fulltext
12 1 - 50 of 70
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf