liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
On asymptotic properties of hyperparameter estimators for kernel-based regularization methods
Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Faculty of Science & Engineering.
Chinese Univ Hong Kong, Peoples R China.
Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, Faculty of Science & Engineering.
2018 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 94, p. 381-395Article in journal (Refereed) Published
Abstract [en]

The kernel-based regularization method has two core issues: kernel design and hyperparameter estimation. In this paper, we focus on the second issue and study the properties of several hyperparameter estimators including the empirical Bayes (EB) estimator, two Steins unbiased risk estimators (SURE) (one related to impulse response reconstruction and the other related to output prediction) and their corresponding Oracle counterparts, with an emphasis on the asymptotic properties of these hyperparameter estimators. To this goal, we first derive and then rewrite the first order optimality conditions of these hyperparameter estimators, leading to several insights on these hyperparameter estimators. Then we show that as the number of data goes to infinity, the two SUREs converge to the best hyperparameter minimizing the corresponding mean square error, respectively, while the more widely used EB estimator converges to another best hyperparameter minimizing the expectation of the EB estimation criterion. This indicates that the two SUREs are asymptotically optimal in the corresponding MSE senses but the EB estimator is not. Surprisingly, the convergence rate of two SUREs is slower than that of the EB estimator, and moreover, unlike the two SUREs, the EB estimator is independent of the convergence rate of Phi(T)Phi/N to its limit, where Phi is the regression matrix and N is the number of data. A Monte Carlo simulation is provided to demonstrate the theoretical results. (C) 2018 Elsevier Ltd. All rights reserved.

Place, publisher, year, edition, pages
PERGAMON-ELSEVIER SCIENCE LTD , 2018. Vol. 94, p. 381-395
Keywords [en]
Kernel-based regularization; Empirical Bayes; Steins unbiased risk estimator; Asymptotic analysis
National Category
Control Engineering
Identifiers
URN: urn:nbn:se:liu:diva-149840DOI: 10.1016/j.automatica.2018.04.035ISI: 000437076500041OAI: oai:DiVA.org:liu-149840DiVA, id: diva2:1236457
Note

Funding Agencies|National Natural Science Foundation of China [61773329, 61603379]; central government of China; Shenzhen Science and Technology Innovation Council [Ji-20170189, Ji-20160207]; Chinese University of Hong Kong, Shenzhen [PF. 01.000249, 2014.0003.23]; Swedish Research Council [2014-5894]; National Key Basic Research Program of China (973 Program) [2014CB845301]; Presidential Fund of the Academy of Mathematics and Systems Science, CAS [2015-hwyxqnrc-mbq]

Available from: 2018-08-02 Created: 2018-08-02 Last updated: 2018-08-22

Open Access in DiVA

The full text will be freely available from 2020-05-29 00:01
Available from 2020-05-29 00:01

Other links

Publisher's full text

Authority records BETA

Mu, BiqiangLjung, Lennart

Search in DiVA

By author/editor
Mu, BiqiangLjung, Lennart
By organisation
Automatic ControlFaculty of Science & Engineering
In the same journal
Automatica
Control Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 72 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf