liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Direct Preference Optimization for Improved Technical WritingAssistance: A Study of How Language Models Can Support the Writing of Technical Documentation at Saab
Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning.
Linköping University, Department of Computer and Information Science, The Division of Statistics and Machine Learning.
2024 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesisAlternative title
En studie i hur språkmodeller kan stödja skrivandet av teknisk dokumentation på Saab (Swedish)
Abstract [en]

This thesis explores the potential of Large Language Models (LLMs) to assist in the technical documentation process at Saab. With the increasing complexity and regulatory demands on such documentation, the objective is to investigate advanced natural language processing techniques as a means of streamlining the creation of technical documentation. Although many standards exist, this thesis particularly focuses on the standard ASD-STE100, Simplified Technical English abbrv. STE, a controlled language for technical documentation. STE's primary aim is to ensure that technical documents are understandable to individuals regardless of their native language or English proficiency. 

The study focuses on the implementation of Direct Preference Optimization (DPO) and Supervised Instruction Fine-Tuning (SIFT) to refine the capabilities of LLMs in producing clear and concise outputs that comply with STE. Through a series of experiments, we investigate the effectiveness of LLMs in interpreting and simplifying technical language, with a particular emphasis on adherence to STE standards. The study utilizes a dataset comprised of target data paired with synthetic source data generated by a LLM. We apply various model training strategies, including zero-shot performance, supervised instruction fine-tuning, and direct preference optimization. We evaluate the various models' output using established quantitative metrics for text simplification and substitute human evaluators with company internal software for evaluating adherence to company standards and STE.

Our findings suggest that while LLMs can significantly contribute to the technical writing process, the choice of training methods and the quality of data play crucial roles in the model's performance. This study shows how LLMs can improve productivity and reduce manual work. It also looks at the problems and suggests ways to make technical documentation automation better in the future.

Place, publisher, year, edition, pages
2024. , p. 69
Keywords [en]
Large Language Models, LLM, Natural Language Processing, NLP, Technical Writing, Simplified Technical English, Direct Preference Optimization, DPO, Supervised Instruction Fine-tuning, LoRA, AI
National Category
Natural Language Processing
Identifiers
URN: urn:nbn:se:liu:diva-205071ISRN: LIU-IDA/LITH-EX-A--24/057--SEOAI: oai:DiVA.org:liu-205071DiVA, id: diva2:1873446
External cooperation
Saab Aeronautics, Mats Hermansson
Subject / course
Computer science
Supervisors
Examiners
Available from: 2024-06-20 Created: 2024-06-19 Last updated: 2025-02-07Bibliographically approved

Open Access in DiVA

fulltext(927 kB)738 downloads
File information
File name FULLTEXT01.pdfFile size 927 kBChecksum SHA-512
e4e865537d589f9bcc18ecfc7f392cee83ed6c4e15420208174d03eda2ebfe6157bad13dbbfaf19e42783a6300d25cbaace6b2f8ed0749594f0bc71af30e66b0
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Bengtsson, HannesHabbe, Patrik
By organisation
The Division of Statistics and Machine Learning
Natural Language Processing

Search outside of DiVA

GoogleGoogle Scholar
Total: 738 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 1435 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf