Open this publication in new window or tab >>2018 (English)In: Automatica, ISSN 0005-1098, E-ISSN 1873-2836, Vol. 94, p. 381-395Article in journal (Refereed) Published
Abstract [en]
The kernel-based regularization method has two core issues: kernel design and hyperparameter estimation. In this paper, we focus on the second issue and study the properties of several hyperparameter estimators including the empirical Bayes (EB) estimator, two Steins unbiased risk estimators (SURE) (one related to impulse response reconstruction and the other related to output prediction) and their corresponding Oracle counterparts, with an emphasis on the asymptotic properties of these hyperparameter estimators. To this goal, we first derive and then rewrite the first order optimality conditions of these hyperparameter estimators, leading to several insights on these hyperparameter estimators. Then we show that as the number of data goes to infinity, the two SUREs converge to the best hyperparameter minimizing the corresponding mean square error, respectively, while the more widely used EB estimator converges to another best hyperparameter minimizing the expectation of the EB estimation criterion. This indicates that the two SUREs are asymptotically optimal in the corresponding MSE senses but the EB estimator is not. Surprisingly, the convergence rate of two SUREs is slower than that of the EB estimator, and moreover, unlike the two SUREs, the EB estimator is independent of the convergence rate of Phi(T)Phi/N to its limit, where Phi is the regression matrix and N is the number of data. A Monte Carlo simulation is provided to demonstrate the theoretical results. (C) 2018 Elsevier Ltd. All rights reserved.
Place, publisher, year, edition, pages
PERGAMON-ELSEVIER SCIENCE LTD, 2018
Keywords
Kernel-based regularization; Empirical Bayes; Steins unbiased risk estimator; Asymptotic analysis
National Category
Control Engineering
Identifiers
urn:nbn:se:liu:diva-149840 (URN)10.1016/j.automatica.2018.04.035 (DOI)000437076500041 ()
Note
Funding Agencies|National Natural Science Foundation of China [61773329, 61603379]; central government of China; Shenzhen Science and Technology Innovation Council [Ji-20170189, Ji-20160207]; Chinese University of Hong Kong, Shenzhen [PF. 01.000249, 2014.0003.23]; Swedish Research Council [2014-5894]; National Key Basic Research Program of China (973 Program) [2014CB845301]; Presidential Fund of the Academy of Mathematics and Systems Science, CAS [2015-hwyxqnrc-mbq]
2018-08-022018-08-022024-01-08