liu.seSearch for publications in DiVA
3738394041424340 of 69
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Analyzing Root Causes and Smells of Test Flakiness by Simulating Resource Usage: A study about how system resource limitations can induce flaky behavior
Linköping University, Department of Computer and Information Science.
Linköping University, Department of Computer and Information Science.
2025 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesisAlternative title
Att Analysera Rotorsaker och Lukter av Testinstabilitet genom att Simulera Resursanvändning : En studie om hur resursbegränsningar i system kan orsaka testinstabilitet (Swedish)
Abstract [en]

Flaky tests, which intermittently pass or fail when no changes have been made to the code, are a significant challenge in regression testing. Such flakiness affects the reliability of test outcomes, forcing developers to debug false alerts. Current research suggests that some test flakiness is associated with computational resource availability. In this thesis, we investigate the impact of computational resource availability, specifically CPU time, on the frequency of test flakiness in an industrial C++ codebase. We conduct experiments by rerunning tests under simulated CPU usage to quantify this impact. Furthermore, we analyze test source code characteristics and known flakiness root causes to provide insight.

The findings reveal that CPU usage significantly influences certain types of flakiness. While concurrency-related flakiness is varyingly affected, flakiness associated with waiting asynchronously is shown to be completely dependent on CPU usage. Additionally, the study suggests that maintainability and thread sleep-related software characteristics could serve as predictors of resource-affected flakiness. These insights highlight the importance of considering computational resources in test design and suggest potential areas for improving test reliability through resource management and code analysis practices.

Place, publisher, year, edition, pages
2025. , p. 67
Keywords [en]
Regression Testing, Flaky Tests, Test Smells, Flaky Test Detection, Test Flakiness Root Causes, Stress Testing, Computational Resources, Resource Availability, Code Metrics
National Category
Computer Sciences
Identifiers
URN: urn:nbn:se:liu:diva-212252ISRN: LIU-IDA/LITH-EX-A--25/001--SEOAI: oai:DiVA.org:liu-212252DiVA, id: diva2:1955392
External cooperation
Ericsson
Subject / course
Computer Engineering
Presentation
2025-01-17, Alan Turing, Linköping, 13:15 (English)
Supervisors
Examiners
Available from: 2025-04-30 Created: 2025-04-30 Last updated: 2025-04-30Bibliographically approved

Open Access in DiVA

fulltext(1299 kB)54 downloads
File information
File name FULLTEXT01.pdfFile size 1299 kBChecksum SHA-512
99b191fac1bba9ae5c9dd8330efb69f471be43053ccd76a77968fea5956a0598c414b24c3b991f5077862d31311fcd8198dcd3ef6ad94a13f5561ef9e10bbaca
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Jonson, MartinTörnqvist, Simon
By organisation
Department of Computer and Information Science
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 54 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 362 hits
3738394041424340 of 69
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf