liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Overtraining, Regularization and Searching for Minimum with Application to Neural Nets
Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
Linköping University, Department of Electrical Engineering, Automatic Control. Linköping University, The Institute of Technology.
1995 (English)In: International Journal of Control, ISSN 0020-7179, E-ISSN 1366-5820, Vol. 62, no 6, 1391-1407 p.Article in journal (Refereed) Published
Abstract [en]

In this paper we discuss the role of criterion minimization as a means for parameter estimation. Most traditional methods, such as maximum likelihood and prediction error identification are based on these principles. However, somewhat surprisingly, it turns out that it is not always ‘optimal’ to try to find the absolute minimum point of the criterion. The reason is that ‘stopped minimization’ (where the iterations have been terminated before the absolute minimum has been reached) has more or less identical properties as using regularization (adding a parametric penalty term). Regularization is known to have beneficial effects on the variance of the parameter estimates and it reduces the ‘variance contribution’ of the misfit. This also explains the concept of ‘overtraining’ in neural nets. How does one know when to terminate the iterations then? A useful criterion would be to stop iterations when the criterion function applied to a validation data set no longer decreases. However, in this paper, we show that applying this technique extensively may lead to the fact that the resulting estimate is an unregularized estimate for the total data set: estimation + validation data.

Place, publisher, year, edition, pages
1995. Vol. 62, no 6, 1391-1407 p.
Keyword [en]
Regularization, Application, Neural nets
National Category
Control Engineering
Identifiers
URN: urn:nbn:se:liu:diva-56345DOI: 10.1080/00207179508921605OAI: oai:DiVA.org:liu-56345DiVA: diva2:318612
Available from: 2010-05-09 Created: 2010-05-07 Last updated: 2017-12-12

Open Access in DiVA

No full text

Other links

Publisher's full text

Authority records BETA

Ljung, Lennart

Search in DiVA

By author/editor
Ljung, Lennart
By organisation
Automatic ControlThe Institute of Technology
In the same journal
International Journal of Control
Control Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 109 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf