liu.seSearch for publications in DiVA
Change search
Link to record
Permanent link

Direct link
BETA
Ljung, Lennart
Alternative names
Publications (10 of 859) Show all publications
Chen, T., Ardeshiri, T., Carli, F. P., Chiuso, A., Ljung, L. & Pillonetto, G. (2016). Maximum entropy properties of discrete-time first-order stable spline kernel. Automatica, 66, 34-38
Open this publication in new window or tab >>Maximum entropy properties of discrete-time first-order stable spline kernel
Show others...
2016 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 66, p. 34-38Article in journal (Refereed) Published
Abstract [en]

The first order stable spline (SS-1) kernel (also known as the tunedcorrelated kernel) is used extensively in regularized system identification, where the impulse response is modeled as a zero-mean Gaussian process whose covariance function is given by well designed and tuned kernels. In this paper, we discuss the maximum entropy properties of this kernel. In particular, we formulate the exact maximum entropy problem solved by the SS-1 kernel without Gaussian and uniform sampling assumptions. Under general sampling assumption, we also derive the special structure of the SS-1 kernel (e.g. its tridiagonal inverse and factorization have closed form expression), also giving to it a maximum entropy covariance completion interpretation.

Keyword
System identification;Regularization method;Kernel structure;Maximum entropy
National Category
Signal Processing
Identifiers
urn:nbn:se:liu:diva-121618 (URN)10.1016/j.automatica.2015.12.009 (DOI)
Available from: 2015-09-28 Created: 2015-09-28 Last updated: 2017-12-01Bibliographically approved
Pillonetto, G., Chen, T., Chiuso, A., De Nicolao, G. & Ljung, L. (2016). Regularized linear system identification using atomic, nuclear and kernel-based norms: The role of the stability constraint. Automatica, 69, 137-149
Open this publication in new window or tab >>Regularized linear system identification using atomic, nuclear and kernel-based norms: The role of the stability constraint
Show others...
2016 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 69, p. 137-149Article in journal (Refereed) Published
Abstract [en]

Inspired by ideas taken from the machine learning literature, new regularization techniques have been recently introduced in linear system identification. In particular, all the adopted estimators solve a regularized least squares problem, differing in the nature of the penalty term assigned to the impulse response. Popular choices include atomic and nuclear norms (applied to Hankel matrices) as well as norms induced by the so called stable spline kernels. In this paper, a comparative study of estimators based on these different types of regularizers is reported. Our findings reveal that stable spline kernels outperform approaches based on atomic and nuclear norms since they suitably embed information on impulse response stability and smoothness. This point is illustrated using the Bayesian interpretation of regularization. We also design a new class of regularizers defined by "integral" versions of stable spline/TC kernels. Under quite realistic experimental conditions, the new estimators outperform classical prediction error methods also when the latter are equipped with an oracle for model order selection. (C) 2016 Elsevier Ltd. All rights reserved.

Place, publisher, year, edition, pages
PERGAMON-ELSEVIER SCIENCE LTD, 2016
Keyword
Linear system identification; Kernel-based regularization; Atomic and nuclear norms; Hankel operator; Lasso; Bayesian interpretation of regularization; Gaussian processes; Reproducing kernel Hilbert spaces
National Category
Control Engineering
Identifiers
urn:nbn:se:liu:diva-130057 (URN)10.1016/j.automatica.2016.02.012 (DOI)000377312800015 ()
Note

Funding Agencies|MIUR FIRB project [RBFR12M3AC]; Progetto di Ateneo [CPDA147754/14]; Linnaeus Center CADICS; Swedish Research Council; ERC advanced grant LEARN [267381]; European Research Council; Swedish Research Council (VR) [2014-5894]

Available from: 2016-07-06 Created: 2016-07-06 Last updated: 2017-11-28
Isaksson, A., Sjöberg, J., Tornqvist, D., Ljung, L. & Kok, M. (2015). Using horizon estimation and nonlinear optimization for grey-box identification. Journal of Process Control, 30, 69-79
Open this publication in new window or tab >>Using horizon estimation and nonlinear optimization for grey-box identification
Show others...
2015 (English)In: Journal of Process Control, ISSN 0959-1524, E-ISSN 1873-2771, Vol. 30, p. 69-79Article in journal (Refereed) Published
Abstract [en]

An established method for grey-box identification is to use maximum-likelihood estimation for the nonlinear case implemented via extended Kalman filtering. In applications of (nonlinear) model predictive control a more and more common approach for the state estimation is to use moving horizon estimation, which employs (nonlinear) optimization directly on a model for a whole batch of data. This paper shows that, in the linear case, horizon estimation may also be used for joint parameter estimation and state estimation, as long as a bias correction based on the Kalman filter is included. For the nonlinear case two special cases are presented where the bias correction can be determined without approximation. A procedure how to approximate the bias correction for general nonlinear systems is also outlined. (C) 2015 Elsevier Ltd. All rights reserved.

Place, publisher, year, edition, pages
Elsevier, 2015
Keyword
System identification; State estimation; Parameter estimation; Optimization; Nonlinear systems; Kalman filtering; Moving horizon estimation; Model predictive control
National Category
Control Engineering
Identifiers
urn:nbn:se:liu:diva-120061 (URN)10.1016/j.jprocont.2014.12.008 (DOI)000356196200007 ()
Note

Funding Agencies|Swedish Foundation for Strategic Research (SSF) - as part of the Process Industry Centre Linkoping (PIC-LI); Swedish Agency for Innovation Systems (VINNOVA) through the ITEA 2 project MODRIO; Linnaeus Center CADICS - Swedish Research Council; ERC advanced grant LEARN - European Research Council [similar to267381]

Available from: 2015-07-06 Created: 2015-07-06 Last updated: 2017-12-04
Pillonetto, G., Dinuzzo, F., Chen, T., De Nicolao, G. & Ljung, L. (2014). Kernel methods in system identification, machine learning and function estimation: A survey. Automatica, 50(3), 657-682
Open this publication in new window or tab >>Kernel methods in system identification, machine learning and function estimation: A survey
Show others...
2014 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 50, no 3, p. 657-682Article in journal (Refereed) Published
Abstract [en]

Most of the currently used techniques for linear system identification are based on classical estimation paradigms coming from mathematical statistics. In particular, maximum likelihood and prediction error methods represent the mainstream approaches to identification of linear dynamic systems, with a long history of theoretical and algorithmic contributions. Parallel to this, in the machine learning community alternative techniques have been developed. Until recently, there has been little contact between these two worlds. The first aim of this survey is to make accessible to the control community the key mathematical tools and concepts as well as the computational aspects underpinning these learning techniques. In particular, we focus on kernel-based regularization and its connections with reproducing kernel Hilbert spaces and Bayesian estimation of Gaussian processes. The second aim is to demonstrate that learning techniques tailored to the specific features of dynamic systems may outperform conventional parametric approaches for identification of stable linear systems.

Place, publisher, year, edition, pages
International Federation of Automatic Control (IFAC), 2014
Keyword
Linear system identification; Prediction error methods; Model complexity selection; Bias-variance trade-off; Kernel-based regularization; Inverse problems; Reproducing kernel Hilbert spaces; Gaussian processes
National Category
Engineering and Technology
Identifiers
urn:nbn:se:liu:diva-106518 (URN)10.1016/j.automatica.2014.01.001 (DOI)000334003500001 ()
Available from: 2014-05-12 Created: 2014-05-09 Last updated: 2017-12-05
Ohlsson, H., Chen, T., Khoshfetratpakazad, S., Ljung, L. & Sastry, S. S. (2014). Scalable anomaly detection in large homogeneous populations. Automatica, 50(5), 1459-1465
Open this publication in new window or tab >>Scalable anomaly detection in large homogeneous populations
Show others...
2014 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 50, no 5, p. 1459-1465Article in journal (Refereed) Published
Abstract [en]

Anomaly detection in large populations is a challenging but highly relevant problem. It is essentially a multi-hypothesis problem, with a hypothesis for every division of the systems into normal and anomalous systems. The number of hypothesis grows rapidly with the number of systems and approximate solutions become a necessity for any problem of practical interest. In this paper we take an optimization approach to this multi-hypothesis problem. It is first shown to be equivalent to a non-convex combinatorial optimization problem and then is relaxed to a convex optimization problem that can be solved distributively on the systems and that stays computationally tractable as the number of systems increase. An interesting property of the proposed method is that it can under certain conditions be shown to give exactly the same result as the combinatorial multi-hypothesis problem and the relaxation is hence tight.

Place, publisher, year, edition, pages
International Federation of Automatic Control (IFAC), 2014
Keyword
Anomaly detection; Outlier detection; Multi-hypothesis testing; Distributed optimization; System identification
National Category
Engineering and Technology
Identifiers
urn:nbn:se:liu:diva-108173 (URN)10.1016/j.automatica.2014.03.008 (DOI)000336779100015 ()
Available from: 2014-06-28 Created: 2014-06-26 Last updated: 2017-12-05
Chen, T., Andersen, M. S., Ljung, L., Chiuso, A. & Pillonetto, G. (2014). System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques. IEEE Transactions on Automatic Control, 59(11), 2933-2945
Open this publication in new window or tab >>System Identification Via Sparse Multiple Kernel-Based Regularization Using Sequential Convex Optimization Techniques
Show others...
2014 (English)In: IEEE Transactions on Automatic Control, ISSN 0018-9286, E-ISSN 1558-2523, Vol. 59, no 11, p. 2933-2945Article in journal (Refereed) Published
Abstract [en]

Model estimation and structure detection with short data records are two issues that receive increasing interests in System Identification. In this paper, a multiple kernel-based regularization method is proposed to handle those issues. Multiple kernels are conic combinations of fixed kernels suitable for impulse response estimation, and equip the kernel-based regularization method with three features. First, multiple kernels can better capture complicated dynamics than single kernels. Second, the estimation of their weights by maximizing the marginal likelihood favors sparse optimal weights, which enables this method to tackle various structure detection problems, e. g., the sparse dynamic network identification and the segmentation of linear systems. Third, the marginal likelihood maximization problem is a difference of convex programming problem. It is thus possible to find a locally optimal solution efficiently by using a majorization minimization algorithm and an interior point method where the cost of a single interior-point iteration grows linearly in the number of fixed kernels. Monte Carlo simulations show that the locally optimal solutions lead to good performance for randomly generated starting points.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2014
Keyword
System identification; regularization; kernel; convex optimization; sparsity; structure detection
National Category
Electrical Engineering, Electronic Engineering, Information Engineering
Identifiers
urn:nbn:se:liu:diva-112818 (URN)10.1109/TAC.2014.2351851 (DOI)000344482500007 ()
Note

Funding Agencies|Linnaeus Center CADICS - Swedish Research Council; ERC advanced grant LEARN [267381]; ERC - European Research Council [291405]; MIUR FIRB project "Learning meets time" [RBFR12M3AC]; European Community [257462]

Available from: 2015-01-08 Created: 2014-12-17 Last updated: 2017-12-05
Ljung, L. & Chen, T. (2013). Convexity Issues in System Identification. In: 10th IEEE International Conference on Control & Automation: . Paper presented at 10th IEEE International Conference on Control & Automation (IEEE ICCA 2013), 12-14 June 2013, Hangzhou, China (pp. 1-9). IEEE
Open this publication in new window or tab >>Convexity Issues in System Identification
2013 (English)In: 10th IEEE International Conference on Control & Automation, IEEE , 2013, p. 1-9Conference paper, Published paper (Refereed)
Abstract [en]

System Identification is about estimating models of dynamical systems from measured input-output data. Its traditional foundation is basic statistical techniques, such as maximum likelihood estimation and asymptotic analysis of bias and variance and the like. Maximum likelihood estimation relies on minimization of criterion functions that typically are non-convex, and may cause numerical search problems. Recent interest in identification algorithms has focused on techniques that are centered around convex formulations. This is partly the result of developments in machine learning and statistical learning theory. The development concerns issues of regularization for sparsity and for better tuned bias/variance trade-offs. It also involves the use of subspace methods as well as nuclear norms as proxies to rank constraints. A quite different route to convexity is to use algebraic techniques manipulate the model parameterizations. This article will illustrate all this recent development.                       

Place, publisher, year, edition, pages
IEEE, 2013
Keyword
System Modelling and Identification, Learning Systems
National Category
Control Engineering
Identifiers
urn:nbn:se:liu:diva-96764 (URN)10.1109/ICCA.2013.6565206 (DOI)978-1-4673-4707-5 (ISBN)
Conference
10th IEEE International Conference on Control & Automation (IEEE ICCA 2013), 12-14 June 2013, Hangzhou, China
Available from: 2013-08-26 Created: 2013-08-26 Last updated: 2016-01-11
Wills, A., Schön, T., Ljung, L. & Ninness, B. (2013). Identification of Hammerstein-Wiener Models. Automatica, 49(1), 70-81
Open this publication in new window or tab >>Identification of Hammerstein-Wiener Models
2013 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 49, no 1, p. 70-81Article in journal (Refereed) Published
Abstract [en]

This paper develops and illustrates a new maximum-likelihood based method for the identification of Hammerstein-Wiener model structures. A central aspect is that a very general situation is considered wherein multivariable data, non-invertible Hammerstein and Wiener nonlinearities, and colored stochastic disturbances both before and after the Wiener nonlinearity are all catered for. The method developed here addresses the blind Wiener estimation problem as a special case.

Place, publisher, year, edition, pages
Elsevier, 2013
Keyword
System identification, Hammerstein, Wiener, Block-oriented models, Nonlinear models, Dynamic systems, Monte Carlo method, Smoothing, Expectation maximization algorithm, Particle methods, Maximum Likelihood
National Category
Control Engineering
Identifiers
urn:nbn:se:liu:diva-89528 (URN)10.1016/j.automatica.2012.09.018 (DOI)000313772600008 ()
Funder
Swedish Research Council
Note

Funding Agencies|Australian Research Council||project Calibrating Nonlinear Dynamical Models|621-2010-5876|CADICS, a Linneaus Center||Swedish Research Council||

Available from: 2013-02-26 Created: 2013-02-26 Last updated: 2017-12-06
Ohlsson, H. & Ljung, L. (2013). Identification of Switched Linear Regression Models using Sum-of-Norms Regularization. Automatica, 49(4), 1045-1050
Open this publication in new window or tab >>Identification of Switched Linear Regression Models using Sum-of-Norms Regularization
2013 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 49, no 4, p. 1045-1050Article in journal (Refereed) Published
Abstract [en]

This paper proposes a general convex framework for the identification of switched linear systems. The proposed framework uses over-parameterization to avoid solving the otherwise combinatorially forbidding identification problem, and takes the form of a least-squares problem with a sum-of-norms regularization, a generalization of the 1-regularization. The regularization constant regulates the complexity and is used to trade off the fit and the number of submodels.

Place, publisher, year, edition, pages
Elsevier, 2013
Keyword
Regularization, System identification, Sum-of-norms, Switched linear systems, Piecewise affine systems
National Category
Control Engineering
Identifiers
urn:nbn:se:liu:diva-92612 (URN)10.1016/j.automatica.2013.01.031 (DOI)000317167700024 ()
Funder
Swedish Research Council
Note

Funding Agencies|Swedish foundation for strategic research in the center MOVIII||Swedish Research Council in the Linnaeus center CADICS||European Research Council|267381|Sweden-America Foundation||Swedish Science Foundation||

Available from: 2013-05-16 Created: 2013-05-14 Last updated: 2017-12-06
Chen, T. & Ljung, L. (2013). Implementation of algorithms for tuning parameters in regularized least squares problems in system identification. Automatica, 49(7), 2213-2220
Open this publication in new window or tab >>Implementation of algorithms for tuning parameters in regularized least squares problems in system identification
2013 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 49, no 7, p. 2213-2220Article in journal (Refereed) Published
Abstract [en]

There has been recently a trend to study linear system identification with high order finite impulse response (FIR) models using the regularized least-squares approach. One key of this approach is to solve the hyper-parameter estimation problem that is usually nonconvex. Our goal here is to investigate implementation of algorithms for solving the hyper-parameter estimation problem that can deal with both large data sets and possibly ill-conditioned computations. In particular, a QR factorization based matrix-inversion-free algorithm is proposed to evaluate the cost function in an efficient and accurate way. It is also shown that the gradient and Hessian of the cost function can be computed based on the same QR factorization. Finally, the proposed algorithm and ideas are verified by Monte-Carlo simulations on a large data-bank of test systems and data sets.

Place, publisher, year, edition, pages
Elsevier, 2013
Keyword
Least squares, Regularization, Empirical Bayes method, Marginal likelihood maximization, QR factorization
National Category
Engineering and Technology
Identifiers
urn:nbn:se:liu:diva-96405 (URN)10.1016/j.automatica.2013.03.030 (DOI)000321233900030 ()
Available from: 2013-08-19 Created: 2013-08-19 Last updated: 2017-12-06
Organisations

Search in DiVA

Show all publications