liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Stability-Informed Initialization of Neural Ordinary Differential Equations
Linköping University, Department of Electrical Engineering, Vehicular Systems. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0001-9075-7477
Linköping University, Department of Electrical Engineering, Vehicular Systems. Linköping University, Faculty of Science & Engineering.
Linköping University, Department of Electrical Engineering, Vehicular Systems. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0003-0808-052X
Linköping University, Department of Electrical Engineering, Vehicular Systems. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0001-7349-1937
2024 (English)In: Proceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024 / [ed] Neil Lawrence, PMLR , 2024, Vol. 235, p. 52903-52914Conference paper, Published paper (Refereed)
Abstract [en]

This paper addresses the training of Neural Ordinary Differential Equations (neural ODEs), and in particular explores the interplay between numerical integration techniques, stability regions, step size, and initialization techniques. It is shown how the choice of integration technique implicitly regularizes the learned model, and how the solver’s corresponding stability region affects training and prediction performance. From this analysis, a stability-informed parameter initialization technique is introduced. The effectiveness of the initialization method is displayed across several learning benchmarks and industrial applications.

Place, publisher, year, edition, pages
PMLR , 2024. Vol. 235, p. 52903-52914
Series
Proceedings of Machine Learning Research, ISSN 2640-3498
National Category
Computational Mathematics
Identifiers
URN: urn:nbn:se:liu:diva-210226OAI: oai:DiVA.org:liu-210226DiVA, id: diva2:1917936
Conference
International Conference on Machine Learning, 21-27 July 2024, Vienna, Austria
Note

Funding: This research was supported by the Strategic Research Area at Linköping-Lund in Information Technology (ELLIIT) and the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. Computations were enabled by the Berzelius resource provided by the Knut and Alice Wallenberg Foundation at the National Supercomputer Centre. The authors would like to thank the reviewers for their insightful comments and suggestions, which have significantly improved the manuscript.

Available from: 2024-12-03 Created: 2024-12-03 Last updated: 2024-12-03Bibliographically approved
In thesis
1. Context-Aware Behavior Prediction for Autonomous Driving
Open this publication in new window or tab >>Context-Aware Behavior Prediction for Autonomous Driving
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Autonomous vehicles (AVs) are set to transform transportation by providing safer, more efficient, and accessible mobility solutions. However, deploying AV systems requires designers to ensure these vehicles can navigate complex, dynamic traffic environments safely and precisely. A vital component of this capability is the ability to predict the behavior of surrounding road users, yet achieving reliable predictions is a complex task. This thesis investigates several challenges in trajectory and intention prediction for autonomous driving, focusing on contextual awareness, probabilistic modeling, and differential motion constraints.

A primary focus of this thesis is context awareness, which includes interaction-aware (agent-to-agent) and environment-aware (road-to-agent) modeling. Early approaches to context awareness involved manually crafting interaction features. While effective, these methods rely on predefined heuristics and often scale poorly as environmental complexity increases. To address these limitations, the thesis adopts a graph-based approach that offers greater flexibility and expressiveness. By constructing relational graphs, graph neural networks can be used to learn agent interactions and environmental context in a data-driven manner. The thesis proposes several context-aware models and provides an extensive evaluation of their mechanisms, highlighting their overall impact on prediction performance.

Another core theme of this thesis is addressing the inherent uncertainty and non-determinism of traffic environments. This involves creating models that provide probabilistic predictions. Given the multimodal nature of road-traffic agent behavior, it is also important to design methods that offer multiple candidate predictions for a single condition, enabling AVs to account for different possible future outcomes. This thesis proposes several context-aware frameworks that leverage probabilistic modeling, illustrating how ensemble methods, mixture density networks, and diffusion-based generative models can be adapted to provide uncertainty estimates and multimodal predictions.

While neural networks are well-suited for capturing the complex dynamics of traffic, they do not inherently ensure that outputs conform to physical laws, which poses risks in safety-critical applications. To address this, the thesis incorporates differential motion constraints into the prediction framework to ensure that predicted trajectories are accurate, physically feasible, and robust to noise. In addition to improving prediction accuracy, it is demonstrated how this integration enhances interpretation, extrapolation, and generalization capabilities.

The use of neural ordinary differential equations is a central component of this thesis, providing a data-driven approach to modeling the motion of agents—such as pedestrians—that are challenging to describe using physical laws or rule-based methods. The thesis investigates their application beyond motion prediction to a wide array of sequence modeling tasks, analyzing the effects of numerical integration techniques, stability regions, and initialization methods on model performance. A key contribution is the proposal of a stability-informed initialization (SII) technique, which significantly enhances model convergence, training stability, and prediction accuracy across various learning benchmarks.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2024. p. 76
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 2419
National Category
Robotics and automation
Identifiers
urn:nbn:se:liu:diva-210214 (URN)10.3384/9789180758956 (DOI)9789180758949 (ISBN)9789180758956 (ISBN)
Public defence
2025-01-10, Ada Lovelace, B-building, Campus Valla, Linköping, 10:15 (English)
Opponent
Supervisors
Note

Funding: This research was supported by the Strategic Research Area at Linköping-Lund in Information Technology (ELLIIT), and the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.

Available from: 2024-12-03 Created: 2024-12-03 Last updated: 2025-02-09Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

https://proceedings.mlr.press/v235/westny24a.html

Authority records

Westny, TheodorMohammadi, ArmanJung, DanielFrisk, Erik

Search in DiVA

By author/editor
Westny, TheodorMohammadi, ArmanJung, DanielFrisk, Erik
By organisation
Vehicular SystemsFaculty of Science & Engineering
Computational Mathematics

Search outside of DiVA

GoogleGoogle Scholar

urn-nbn

Altmetric score

urn-nbn
Total: 134 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf