liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Contract-based Validation of Conceptual Design Bugs for Engineering Complex Machine Learning Software
Linköping University, Department of Computer and Information Science, Software and Systems. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0001-8482-3917
2024 (English)In: ACM/IEEE 27TH INTERNATIONAL CONFERENCE ON MODEL DRIVEN ENGINEERING LANGUAGES AND SYSTEMS: COMPANION PROCEEDINGS, MODELS 2024, ASSOC COMPUTING MACHINERY , 2024, p. 155-161Conference paper, Published paper (Refereed)
Abstract [en]

Context. Modern software systems increasingly commonly contain one or multiple machine learning (ML) components. Current development practices are generally on a trial-and-error basis, posing a significant risk of introducing bugs. One type of bug is the "conceptual design bug," referring to a misunderstanding between the properties of input data and prerequisites imposed by ML algorithms (e.g., using unscaled data in a scale-sensitive algorithm). These bugs are challenging to test at design time, causing problems at runtime through crashes, noticeably poor model performance, or not at all, threatening the system's robustness and transparency. Objective. In this work, I propose the line of research I intend to pursue during my PhD, addressing conceptual design bugs in complex ML software from a prevention-oriented perspective. I intend to build open-source tooling for ML engineers that can be used to detect conceptual design bugs, enabling them to make quality assurances about their system design's robustness. Approach. We need to understand conceptual bugs beyond the status quo, identifying their types, prevalence, impacts, and structural elements in the code. We operationalize this knowledge into a tool that detects them at design time, allowing ML engineers to resolve them before running their code and wasting resources. We anticipate this tool will leverage contract-based validation applied to partial ML software models. Evaluation. We plan to evaluate the built tool two-fold using professional (industrial) ML software. First, we will study its effectiveness regarding bug detection at design time, identifying whether it fulfills its functional objective. Second, we will study its usability, identifying whether ML engineers benefit when tools like this are introduced into their ML engineering workflow.

Place, publisher, year, edition, pages
ASSOC COMPUTING MACHINERY , 2024. p. 155-161
Keywords [en]
Machine Learning; Software Bugs; Software Design; Knowledge Mining; Software Contracts; Empirical Software Engineering
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:liu:diva-210353DOI: 10.1145/3652620.3688201ISI: 001351589800028ISBN: 9798400706226 (print)OAI: oai:DiVA.org:liu-210353DiVA, id: diva2:1920022
Conference
ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings (MODELS), Linz, AUSTRIA, sep 22-27, 2024
Note

Funding Agencies|Wallenberg AI, Autonomous Systems and Software Program (WASP) - Knut and Alice Wallenberg Foundation

Available from: 2024-12-10 Created: 2024-12-10 Last updated: 2024-12-10

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Search in DiVA

By author/editor
Meijer, Willem
By organisation
Software and SystemsFaculty of Science & Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 66 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf