liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Context-Aware Behavior Prediction for Autonomous Driving
Linköping University, Department of Electrical Engineering, Vehicular Systems. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0001-9075-7477
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Autonomous vehicles (AVs) are set to transform transportation by providing safer, more efficient, and accessible mobility solutions. However, deploying AV systems requires designers to ensure these vehicles can navigate complex, dynamic traffic environments safely and precisely. A vital component of this capability is the ability to predict the behavior of surrounding road users, yet achieving reliable predictions is a complex task. This thesis investigates several challenges in trajectory and intention prediction for autonomous driving, focusing on contextual awareness, probabilistic modeling, and differential motion constraints.

A primary focus of this thesis is context awareness, which includes interaction-aware (agent-to-agent) and environment-aware (road-to-agent) modeling. Early approaches to context awareness involved manually crafting interaction features. While effective, these methods rely on predefined heuristics and often scale poorly as environmental complexity increases. To address these limitations, the thesis adopts a graph-based approach that offers greater flexibility and expressiveness. By constructing relational graphs, graph neural networks can be used to learn agent interactions and environmental context in a data-driven manner. The thesis proposes several context-aware models and provides an extensive evaluation of their mechanisms, highlighting their overall impact on prediction performance.

Another core theme of this thesis is addressing the inherent uncertainty and non-determinism of traffic environments. This involves creating models that provide probabilistic predictions. Given the multimodal nature of road-traffic agent behavior, it is also important to design methods that offer multiple candidate predictions for a single condition, enabling AVs to account for different possible future outcomes. This thesis proposes several context-aware frameworks that leverage probabilistic modeling, illustrating how ensemble methods, mixture density networks, and diffusion-based generative models can be adapted to provide uncertainty estimates and multimodal predictions.

While neural networks are well-suited for capturing the complex dynamics of traffic, they do not inherently ensure that outputs conform to physical laws, which poses risks in safety-critical applications. To address this, the thesis incorporates differential motion constraints into the prediction framework to ensure that predicted trajectories are accurate, physically feasible, and robust to noise. In addition to improving prediction accuracy, it is demonstrated how this integration enhances interpretation, extrapolation, and generalization capabilities.

The use of neural ordinary differential equations is a central component of this thesis, providing a data-driven approach to modeling the motion of agents—such as pedestrians—that are challenging to describe using physical laws or rule-based methods. The thesis investigates their application beyond motion prediction to a wide array of sequence modeling tasks, analyzing the effects of numerical integration techniques, stability regions, and initialization methods on model performance. A key contribution is the proposal of a stability-informed initialization (SII) technique, which significantly enhances model convergence, training stability, and prediction accuracy across various learning benchmarks.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2024. , p. 76
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 2419
National Category
Robotics and automation
Identifiers
URN: urn:nbn:se:liu:diva-210214DOI: 10.3384/9789180758956ISBN: 9789180758949 (print)ISBN: 9789180758956 (electronic)OAI: oai:DiVA.org:liu-210214DiVA, id: diva2:1917579
Public defence
2025-01-10, Ada Lovelace, B-building, Campus Valla, Linköping, 10:15 (English)
Opponent
Supervisors
Note

Funding: This research was supported by the Strategic Research Area at Linköping-Lund in Information Technology (ELLIIT), and the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.

Available from: 2024-12-03 Created: 2024-12-03 Last updated: 2025-02-09Bibliographically approved
List of papers
1. Vehicle Behavior Prediction and Generalization Using Imbalanced Learning Techniques
Open this publication in new window or tab >>Vehicle Behavior Prediction and Generalization Using Imbalanced Learning Techniques
2021 (English)In: 24th IEEE International Intelligent Transportation Systems Conference (ITSC), 19-22 Sept. 2021, Institute of Electrical and Electronics Engineers (IEEE), 2021, p. 2003-2010Conference paper, Published paper (Refereed)
Abstract [en]

The use of learning-based methods for vehicle behavior prediction is a promising research topic. However, many publicly available data sets suffer from class distribution skews which limits learning performance if not addressed. This paper proposes an interaction-aware prediction model consisting of an LSTM autoencoder and SVM classifier. Additionally, an imbalanced learning technique, the multiclass balancing ensemble is proposed. Evaluations show that the method enhances model performance, resulting in improved classification accuracy. Good generalization properties of learned models are important and therefore a generalization study is done where models are evaluated on unseen traffic data with dissimilar traffic behavior stemming from different road configurations. This is realized by using two distinct highway traffic recordings, the publicly available NGSIM US-101 and I80 data sets. Moreover, methods for encoding structural and static features into the learning process for improved generalization are evaluated. The resulting methods show substantial improvements in classification as well as generalization performance.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2021
Keywords
Behavior Prediction, Machine Learning, Autonomous Vehicles, Robotics
National Category
Computer Sciences Robotics and automation
Identifiers
urn:nbn:se:liu:diva-180481 (URN)10.1109/ITSC48978.2021.9564948 (DOI)000841862502003 ()2-s2.0-85118442722 (Scopus ID)9781728191423 (ISBN)9781728191430 (ISBN)
Conference
IEEE International Conference on Intelligent Transportation Systems - ITSC2021, Indianapolis, IN, USA, 19-22 Sept. 2021
Funder
ELLIIT - The Linköping‐Lund Initiative on IT and Mobile Communications, 101456
Note

Funding: Strategic Reseach Area at Linkoping-Lund in Information Technology (ELLIIT); Wallenberg AI, Autonomous Systems and Software Program (WASP) - Knut and Alice Wallenberg Foundation

Available from: 2022-01-19 Created: 2022-01-19 Last updated: 2025-02-05
2. MTP-GO: Graph-Based Probabilistic Multi-Agent Trajectory Prediction With Neural ODEs
Open this publication in new window or tab >>MTP-GO: Graph-Based Probabilistic Multi-Agent Trajectory Prediction With Neural ODEs
2023 (English)In: IEEE Transactions on Intelligent Vehicles, ISSN 2379-8858, E-ISSN 2379-8904, Vol. 8, no 9, p. 4223-4236Article in journal (Refereed) Published
Abstract [en]

Enabling resilient autonomous motion planning requires robust predictions of surrounding road users’ future behavior. In response to this need and the associated challenges, we introduce our model titled MTP-GO. The model encodes the scene using temporal graph neural networks to produce the inputs to an underlying motion model. The motion model is implemented using neural ordinary differential equations where the state-transition functions are learned with the rest of the model. Multimodal probabilistic predictions are obtained by combining the concept of mixture density networks and Kalman filtering. The results illustrate the predictive capabilities of the proposed model across various data sets, outperforming several state-of-the-art methods on a number of metrics.

Place, publisher, year, edition, pages
IEEE, 2023
Keywords
Predictive models;Trajectory;Computational modeling;Mathematical models;Data models;Roads;Behavioral sciences;Graph neural networks;neural ODEs;trajectory prediction
National Category
Computer graphics and computer vision
Identifiers
urn:nbn:se:liu:diva-203164 (URN)10.1109/TIV.2023.3282308 (DOI)2-s2.0-8516155373 (Scopus ID)
Note

Fundng agencies: the Strategic Research Area at Linköping-Lund in Information Technology (ELLIIT), in part by the Swedish Research Council through the Project Handling Uncertainty in Machine Learning Systems under Grant 2020-04122, and in part by the Knutand Alice Wallenberg Foundation through Wallenberg AI, Autonomous Systemsand Software Program (WASP)

Available from: 2024-04-30 Created: 2024-04-30 Last updated: 2025-06-26
3. Evaluation of Differentially Constrained Motion Models for Graph-Based Trajectory Prediction
Open this publication in new window or tab >>Evaluation of Differentially Constrained Motion Models for Graph-Based Trajectory Prediction
2023 (English)In: 2023 IEEE INTELLIGENT VEHICLES SYMPOSIUM, IV, IEEE , 2023Conference paper, Published paper (Refereed)
Abstract [en]

Given their flexibility and encouraging performance, deep-learning models are becoming standard for motion prediction in autonomous driving. However, with great flexibility comes a lack of interpretability and possible violations of physical constraints. Accompanying these data-driven methods with differentially-constrained motion models to provide physically feasible trajectories is a promising future direction. The foundation for this work is a previously introduced graph-neural-network-based model, MTP-GO. The neural network learns to compute the inputs to an underlying motion model to provide physically feasible trajectories. This research investigates the performance of various motion models in combination with numerical solvers for the prediction task. The study shows that simpler models, such as low-order integrator models, are preferred over more complex, e.g., kinematic models, to achieve accurate predictions. Further, the numerical solver can have a substantial impact on performance, advising against commonly used first-order methods like Euler forward. Instead, a second-order method like Heuns can greatly improve predictions.

Place, publisher, year, edition, pages
IEEE, 2023
Series
IEEE Intelligent Vehicles Symposium, ISSN 1931-0587
National Category
Control Engineering
Identifiers
urn:nbn:se:liu:diva-197928 (URN)10.1109/IV55152.2023.10186615 (DOI)001042247300083 ()9798350346916 (ISBN)9798350346923 (ISBN)
Conference
34th IEEE Intelligent Vehicles Symposium (IV), Anchorage, AK, jun 04-07, 2023
Note

Funding Agencies|Strategic Research Area at Linkoping-Lund in Information Technology (ELLIIT); Swedish Research Council via the project Handling Uncertainty in Machine Learning Systems [2020-04122]; Wallenberg AI, Autonomous Systems and Software Program (WASP) - Knut and Alice Wallenberg Foundation

Available from: 2023-09-21 Created: 2023-09-21 Last updated: 2024-12-03
4. Diffusion-Based Environment-Aware Trajectory Prediction
Open this publication in new window or tab >>Diffusion-Based Environment-Aware Trajectory Prediction
(English)Manuscript (preprint) (Other academic)
Abstract [en]

The ability to predict the future trajectories of traffic participants is crucial for the safe and efficient operation of autonomous vehicles. In this paper, a diffusion-based generative model for multi-agent trajectory prediction is proposed. The model is capable of capturing the complex interactions between traffic participants and the environment, accurately learning the multimodal nature of the data. The effectiveness of the approach is assessed on large-scale datasets of real-world traffic scenarios, showing that our model outperforms several well-established methods in terms of prediction accuracy. By the incorporation of differential motion constraints on the model output, we illustrate that our model is capable of generating a diverse set of realistic future trajectories. Through the use of an interaction-aware guidance signal, we further demonstrate that the model can be adapted to predict the behavior of less cooperative agents, emphasizing its practical applicability under uncertain traffic conditions.

Keywords
Trajectory Prediction, Generative Modeling, Autonomous Driving
National Category
Robotics and automation
Identifiers
urn:nbn:se:liu:diva-210232 (URN)10.48550/arXiv.2403.11643 (DOI)
Note

This a preprint posted 18 Mars 2024 at arXiv, https://arxiv.org/abs/2403.11643

This version is not peer-reviewed.

Available from: 2024-12-04 Created: 2024-12-04 Last updated: 2025-02-09Bibliographically approved
5. Stability-Informed Initialization of Neural Ordinary Differential Equations
Open this publication in new window or tab >>Stability-Informed Initialization of Neural Ordinary Differential Equations
2024 (English)In: Proceedings of the 41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024 / [ed] Neil Lawrence, PMLR , 2024, Vol. 235, p. 52903-52914Conference paper, Published paper (Refereed)
Abstract [en]

This paper addresses the training of Neural Ordinary Differential Equations (neural ODEs), and in particular explores the interplay between numerical integration techniques, stability regions, step size, and initialization techniques. It is shown how the choice of integration technique implicitly regularizes the learned model, and how the solver’s corresponding stability region affects training and prediction performance. From this analysis, a stability-informed parameter initialization technique is introduced. The effectiveness of the initialization method is displayed across several learning benchmarks and industrial applications.

Place, publisher, year, edition, pages
PMLR, 2024
Series
Proceedings of Machine Learning Research, ISSN 2640-3498
National Category
Computational Mathematics
Identifiers
urn:nbn:se:liu:diva-210226 (URN)
Conference
International Conference on Machine Learning, 21-27 July 2024, Vienna, Austria
Note

Funding: This research was supported by the Strategic Research Area at Linköping-Lund in Information Technology (ELLIIT) and the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation. Computations were enabled by the Berzelius resource provided by the Knut and Alice Wallenberg Foundation at the National Supercomputer Centre. The authors would like to thank the reviewers for their insightful comments and suggestions, which have significantly improved the manuscript.

Available from: 2024-12-03 Created: 2024-12-03 Last updated: 2024-12-03Bibliographically approved

Open Access in DiVA

fulltext(17709 kB)1261 downloads
File information
File name FULLTEXT01.pdfFile size 17709 kBChecksum SHA-512
7c2b7cab4e19c9151eea0aff203afa1d7c71db1c9441772cedca05b5c0b9e5eb34b37972c204399b2e216ced591d38d14b6e5f2b4e688b828980c25d16eeeb3b
Type fulltextMimetype application/pdf
Order online >>

Other links

Publisher's full text

Authority records

Westny, Theodor

Search in DiVA

By author/editor
Westny, Theodor
By organisation
Vehicular SystemsFaculty of Science & Engineering
Robotics and automation

Search outside of DiVA

GoogleGoogle Scholar
Total: 1263 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 3035 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf