liu.seSearch for publications in DiVA
3334353637383936 of 77
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Evaluating Generative AI's Applicability in Software Testing: User study and quality evaluation of unit test generation
Linköping University, Department of Computer and Information Science.
Linköping University, Department of Computer and Information Science.
2025 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

This thesis explores the current capabilities and perceptions of generative AI tools for software testing in an industrial environment, focusing on unit test generation. Despite the potential for generative AI to automate test generation and improve efficiency, the study reveals challenges related to output quality, compilability, and practical integration barriers. Using a commercial coding assistant, we generate unit tests for an industrial code base. We then evaluate the generated tests, revealing a compilation rate of 36.09% and emphasizing compilability as the primary obstacle in generating high-quality tests. To reveal the perceived benefits and challenges of using generative AI in software testing, we conduct interviews with engineers to gather qualitative data and perform a thematic analysis. The analysis reveals perceived benefits such as time savings, efficiency improvements, and organizational support and encouragement. It also highlights challenges related to security and data confidentiality concerns, validation requirements, and output quality and reliability. While engineers recognize the theoretical potential of generative AI tools in unit test generation, the adoption of such tools in industry remains limited, with some viewing it as a misapplication of the technology. This research contributes to more informed decision-making for companies considering integrating generative AI tools into their software testing workflows.

Place, publisher, year, edition, pages
2025. , p. 59
Keywords [en]
Artificial Intelligence, AI, Large Language Models, LLMs, Coding Assistant, User Study, Software Testing, Unit Test, Unit Test Generation, Generative AI, Benefits, Challenges, Thematic Analysis
National Category
Artificial Intelligence
Identifiers
URN: urn:nbn:se:liu:diva-219190ISRN: LIU-IDA/LITH-EX-A--25/021--SEOAI: oai:DiVA.org:liu-219190DiVA, id: diva2:2010682
External cooperation
Undisclosed company
Subject / course
Computer Engineering
Presentation
2025-06-16, Donald Knuth, Linköping, 13:00 (English)
Supervisors
Examiners
Available from: 2025-11-28 Created: 2025-11-02 Last updated: 2025-11-28Bibliographically approved

Open Access in DiVA

fulltext(1290 kB)41 downloads
File information
File name FULLTEXT01.pdfFile size 1290 kBChecksum SHA-512
fd509fdb710d3423afdcf4ac96c3479360e9ea88e6388d91703c16d8f9d5fe5972a0366e8b1e1ae3c53188551f324732d0f357af0530cdfe33a2cf041388d74f
Type fulltextMimetype application/pdf

By organisation
Department of Computer and Information Science
Artificial Intelligence

Search outside of DiVA

GoogleGoogle Scholar
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 268 hits
3334353637383936 of 77
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf