Evaluation of LLMs within the FMI simulation standard
2026 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE credits
Student thesisAlternative title
Utvärdering av stora språkmodeller i simuleringsstandarden FMI (Swedish)
Abstract [en]
Simulation of complex systems plays a critical role in the development process and personnel training in industries such as aerospace. The Functional Mock-up Interface (FMI) standard is used for co-simulation, where different models can exchange information and be integrated through a standardized interface. The standard was created with deterministic, fast, and self-contained models in mind. With the rapid development and improvements of LLMs, there could be several benefits of integrating LLMs into the standard. However, LLMs differ in many aspects from the physics-based models that the FMI standard was built for.
This thesis investigated the feasibility of using LLMs in simulations using the FMI standard. Different approaches were evaluated, how the FMI standard enables and constrains model interactions for LLMs was explored, and whether LLMs could comply with co-simulation requirements for models. The questions were evaluated using two use cases. The first consists of feeding the model multiple errors from an airplane, with the task of finding the root cause of the errors. The second task involved generating natural language by providing pilot training instructions based on a specific airplane scenario, with information and scenarios from pilot handbooks.
Results showed that not all requirements and desired attributes could be fulfilled at the same time, and would depend on the chosen approach. An external model could provide relatively high accuracy without the need for expensive local hardware. However, it relies on external services that will introduce latency and external dependencies to the simulation. Local approaches could instead be independent of external services, but could result in a large model package and need significantly better local hardware to get acceptable performance. Therefore, depending on the requirements, different approaches and models should be considered. Trade-offs will be made, primarily between accuracy and latency, since larger models generally have higher accuracy and higher latency. The second use case demonstrated how accuracy can be increased with a RAG system, although latency would once again increase. Therefore, several trade-offs would need to be made when choosing an integration approach.
Place, publisher, year, edition, pages
2026. , p. 39
Keywords [en]
llm, fmi, large language model, AI, simulation
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:liu:diva-225703ISRN: LIU-IDA/LITH-EX-A--26/086--SEOAI: oai:DiVA.org:liu-225703DiVA, id: diva2:2079670
External cooperation
Saab Aeronautics
Subject / course
Computer Engineering
Presentation
2026-06-12, Kurt Gödel, 14:16 (English)
Supervisors
Examiners
2026-06-302026-06-252026-06-30Bibliographically approved