Automated Unit Test Generation for the Google Test Framework Using Large Language Models: An Industrial Case Study
2024 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE credits
Student thesisAlternative title
Automatiserad generering av enhetstester för ramverket Google Test med hjälp av Large Language models : En Fallstudie i Industrin (Swedish)
Abstract [en]
Unit tests serve a critical role in software development in ensuring quality and pre-dictability for components of code. While unit tests are important, they are a resourceheavy part of the development life cycle. Hence, the automation of unit tests is an inten-sive research area. This paper looks at the use of Large Language Models (LLMs) to achievethis task by looking at both fine-tuning and in-context learning of various sizes in order toautomatically generate unit tests given a function, and seeing how they perform in terms ofa similarity score. In order to achieve this, this thesis proposes an implementation pipelinewhich consist of creating a dataset that maps functions to unit tests, training an LLM, Wiz-ardCoder, of various sizes on a GPU-cluster, running inference, and evaluating the qualityof the generated tests. These results that the fine-tuned models consistently outperformthe models using in-context learning, independent of size. Moreover, within the scope offine-tuned models, the models using more parameters performed slightly better than theirsmaller counterparts.
Place, publisher, year, edition, pages
2024. , p. 37
Keywords [en]
Machine Learning, Unit Tests, Test generation, C++, Large Language Models, LLM, Similarity
National Category
Telecommunications
Identifiers
URN: urn:nbn:se:liu:diva-205941ISRN: LIU-IDA/LITH-EX-A--24/067--SEOAI: oai:DiVA.org:liu-205941DiVA, id: diva2:1884076
External cooperation
Camilla Bodin, Ericsson AB
Subject / course
Computer Engineering
Presentation
2024-06-14, John von Neumann, 58183, Linköping, 13:15 (English)
Supervisors
Examiners
2024-12-122024-07-122024-12-12Bibliographically approved