A dense initialization for limited-memory quasi-Newton methods
2019 (English)In: Computational Optimization and Applications, ISSN 0926-6003, Vol. 74, no 1, p. 121-142Article in journal (Other academic) Published
Abstract [en]
We consider a family of dense initializations for limited-memory quasi-Newton methods. The proposed initialization exploits an eigendecomposition-based separation of the full space into two complementary subspaces, assigning a different initialization parameter to each subspace. This family of dense initializations is proposed in the context of a limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) trust-region method that makes use of a shape-changing norm to define each subproblem. As with L-BFGS methods that traditionally use diagonal initialization, the dense initialization and the sequence of generated quasi-Newton matrices are never explicitly formed. Numerical experiments on the CUTEst test set suggest that this initialization together with the shape-changing trust-region method outperforms other L-BFGS methods for solving general nonconvex unconstrained optimization problems. While this dense initialization is proposed in the context of a special trust-region method, it has broad applications for more general quasi-Newton trust-region and line search methods. In fact, this initialization is suitable for use with any quasi-Newton update that admits a compact representation and, in particular, any member of the Broyden class of updates.
Place, publisher, year, edition, pages
Springer, 2019. Vol. 74, no 1, p. 121-142
Keywords [en]
Large-scale nonlinear optimization, limited-memory quasi-Newton methods, trust-region methods, quasi-Newton matrices, shape-changing norm.
National Category
Computational Mathematics
Identifiers
URN: urn:nbn:se:liu:diva-143315DOI: 10.1007/s10589-019-00112-xISI: 000476600200005OAI: oai:DiVA.org:liu-143315DiVA, id: diva2:1162458
Note
Funding agencies: NSF [CMMI-1334042, CMMI-1333326, IIS-1741490, IIS-1741264]
2017-12-042017-12-042019-08-12Bibliographically approved