liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Automated bug assignment: Ensemble-based machine learning in large scale industrial contexts
Linköping University, Department of Computer and Information Science. Linköping University, Faculty of Science & Engineering. Ericsson AB, Sweden.
Lund University, Sweden.
KTH Royal Institute Technology, Sweden; University of Calif Berkeley, CA 94720 USA.
Linköping University, Department of Computer and Information Science, Software and Systems. Linköping University, Faculty of Science & Engineering.
Show others and affiliations
2016 (English)In: Journal of Empirical Software Engineering, ISSN 1382-3256, E-ISSN 1573-7616, Vol. 21, no 4, p. 1533-1578Article in journal (Refereed) Published
Resource type
Text
Abstract [en]

Bug report assignment is an important part of software maintenance. In particular, incorrect assignments of bug reports to development teams can be very expensive in large software development projects. Several studies propose automating bug assignment techniques using machine learning in open source software contexts, but no study exists for large-scale proprietary projects in industry. The goal of this study is to evaluate automated bug assignment techniques that are based on machine learning classification. In particular, we study the state-of-the-art ensemble learner Stacked Generalization (SG) that combines several classifiers. We collect more than 50,000 bug reports from five development projects from two companies in different domains. We implement automated bug assignment and evaluate the performance in a set of controlled experiments. We show that SG scales to large scale industrial application and that it outperforms the use of individual classifiers for bug assignment, reaching prediction accuracies from 50 % to 89 % when large training sets are used. In addition, we show how old training data can decrease the prediction accuracy of bug assignment. We advice industry to use SG for bug assignment in proprietary contexts, using at least 2,000 bug reports for training. Finally, we highlight the importance of not solely relying on results from cross-validation when evaluating automated bug assignment.

Place, publisher, year, edition, pages
SPRINGER , 2016. Vol. 21, no 4, p. 1533-1578
Keywords [en]
Machine learning; Ensemble learning; Classification; Bug reports; Bug assignment; Industrial scale; Large scale
National Category
Software Engineering
Identifiers
URN: urn:nbn:se:liu:diva-130374DOI: 10.1007/s10664-015-9401-9ISI: 000379060700004OAI: oai:DiVA.org:liu-130374DiVA, id: diva2:952818
Note

Funding Agencies|Industrial Excellence Center EASE Embedded Applications Software Engineering

Available from: 2016-08-15 Created: 2016-08-05 Last updated: 2018-05-17
In thesis
1. Machine Learning-Based Bug Handling in Large-Scale Software Development
Open this publication in new window or tab >>Machine Learning-Based Bug Handling in Large-Scale Software Development
2018 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

This thesis investigates the possibilities of automating parts of the bug handling process in large-scale software development organizations. The bug handling process is a large part of the mostly manual, and very costly, maintenance of software systems. Automating parts of this time consuming and very laborious process could save large amounts of time and effort wasted on dealing with bug reports. In this thesis we focus on two aspects of the bug handling process, bug assignment and fault localization. Bug assignment is the process of assigning a newly registered bug report to a design team or developer. Fault localization is the process of finding where in a software architecture the fault causing the bug report should be solved. The main reason these tasks are not automated is that they are considered hard to automate, requiring human expertise and creativity. This thesis examines the possi- bility of using machine learning techniques for automating at least parts of these processes. We call these automated techniques Automated Bug Assignment (ABA) and Automatic Fault Localization (AFL), respectively. We treat both of these problems as classification problems. In ABA, the classes are the design teams in the development organization. In AFL, the classes consist of the software components in the software architecture. We focus on a high level fault localization that it is suitable to integrate into the initial support flow of large software development organizations.

The thesis consists of six papers that investigate different aspects of the AFL and ABA problems. The first two papers are empirical and exploratory in nature, examining the ABA problem using existing machine learning techniques but introducing ensembles into the ABA context. In the first paper we show that, like in many other contexts, ensembles such as the stacked generalizer (or stacking) improves classification accuracy compared to individual classifiers when evaluated using cross fold validation. The second paper thor- oughly explore many aspects such as training set size, age of bug reports and different types of evaluation of the ABA problem in the context of stacking. The second paper also expands upon the first paper in that the number of industry bug reports, roughly 50,000, from two large-scale industry software development contexts. It is still as far as we are aware, the largest study on real industry data on this topic to this date. The third and sixth papers, are theoretical, improving inference in a now classic machine learning tech- nique for topic modeling called Latent Dirichlet Allocation (LDA). We show that, unlike the currently dominating approximate approaches, we can do parallel inference in the LDA model with a mathematically correct algorithm, without sacrificing efficiency or speed. The approaches are evaluated on standard research datasets, measuring various aspects such as sampling efficiency and execution time. Paper four, also theoretical, then builds upon the LDA model and introduces a novel supervised Bayesian classification model that we call DOLDA. The DOLDA model deals with both textual content and, structured numeric, and nominal inputs in the same model. The approach is evaluated on a new data set extracted from IMDb which have the structure of containing both nominal and textual data. The model is evaluated using two approaches. First, by accuracy, using cross fold validation. Second, by comparing the simplicity of the final model with that of other approaches. In paper five we empirically study the performance, in terms of prediction accuracy, of the DOLDA model applied to the AFL problem. The DOLDA model was designed with the AFL problem in mind, since it has the exact structure of a mix of nominal and numeric inputs in combination with unstructured text. We show that our DOLDA model exhibits many nice properties, among others, interpretability, that the research community has iden- tified as missing in current models for AFL.

Place, publisher, year, edition, pages
Linköping: Linköping University Electronic Press, 2018. p. 120
Series
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 1936
Keywords
machine learning, bug reports, large scale software development
National Category
Engineering and Technology Software Engineering
Identifiers
urn:nbn:se:liu:diva-147059 (URN)10.3384/diss.diva-147059 (DOI)9789176853061 (ISBN)
Public defence
2018-06-12, Ada Lovelace, Hus B, Campus Valla, Linköping, 13:15 (English)
Opponent
Supervisors
Available from: 2018-05-17 Created: 2018-05-17 Last updated: 2018-05-29Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full text

Authority records BETA

Jonsson, LeifSandahl, Kristian

Search in DiVA

By author/editor
Jonsson, LeifSandahl, Kristian
By organisation
Department of Computer and Information ScienceFaculty of Science & EngineeringSoftware and Systems
In the same journal
Journal of Empirical Software Engineering
Software Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 265 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf