liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
A Hybrid Sobolev Gradient Method for Learning NODEs
Linköping University, Department of Science and Technology, Physics, Electronics and Mathematics. Linköping University, Faculty of Science & Engineering.
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0002-9217-9997
Linköping University, Department of Science and Technology, Physics, Electronics and Mathematics. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0001-8445-0129
Linköping University, Faculty of Science & Engineering. Linköping University, Department of Science and Technology, Physics, Electronics and Mathematics.ORCID iD: 0000-0001-9066-7922
Show others and affiliations
2024 (English)In: Operations Research Forum, E-ISSN 2662-2556, Vol. 5, article id 91Article in journal (Refereed) Published
Abstract [en]

The inverse problem of supervised reconstruction of depth-variable (time-dependent) parameters in ordinary differential equations is considered, with the typical application of finding weights of a neural ordinary differential equation (NODE) for a residual network with time continuous layers. The differential equation is treated as an abstract and isolated entity, termed a standalone NODE (sNODE), to facilitate for a wide range of applications. The proposed parameter reconstruction is performed by minimizing a cost functional covering a variety of loss functions and penalty terms. Regularization via penalty terms is incorporated to enhance ethical and trustworthy AI formulations. A nonlinear conjugate gradient mini-batch optimization scheme (NCG) is derived for the training having the benefit of including a sensitivity problem. The model (differential equation)-based approach is thus combined with a data-driven learning procedure. Mathematical properties are stated for the differential equation and the cost functional. The adjoint problem needed is derived together with the sensitivity problem. The sensitivity problem itself can estimate changes in the output under perturbation of the trained parameters. To preserve smoothness during the iterations, the Sobolev gradient is calculated and incorporated. Numerical results are included to validate the procedure for a NODE and synthetic datasets and compared with standard gradient approaches. For stability, using the sensitivity problem, a strategy for adversarial attacks is constructed, and it is shown that the given method with Sobolev gradients is more robust than standard approaches for parameter identification.

Place, publisher, year, edition, pages
Switzerland: Springer Nature, 2024. Vol. 5, article id 91
Keywords [en]
Adversarial attacks, Deep learning, Inverse problems, Neural ordinary differential equations, Sobolev gradient
National Category
Mathematics Computer Sciences
Identifiers
URN: urn:nbn:se:liu:diva-208091DOI: 10.1007/s43069-024-00377-xScopus ID: 2-s2.0-85205866958OAI: oai:DiVA.org:liu-208091DiVA, id: diva2:1902924
Available from: 2024-10-02 Created: 2024-10-02 Last updated: 2024-12-12Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Baravdish, GeorgeEilertsen, GabrielJaroudi, RymJohansson, TomasMalý, LukášUnger, Jonas

Search in DiVA

By author/editor
Baravdish, GeorgeEilertsen, GabrielJaroudi, RymJohansson, TomasMalý, LukášUnger, Jonas
By organisation
Physics, Electronics and MathematicsFaculty of Science & EngineeringMedia and Information Technology
MathematicsComputer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 95 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf