liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Classifying the classifier: dissecting the weight space of neural networks
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0002-9217-9997
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0002-5220-633X
Institute of Media Informatics, Ulm University, Germany.
Linköping University, Department of Science and Technology, Media and Information Technology. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0002-7765-1747
Show others and affiliations
2020 (English)In: Proceedings of the 24th European Conference on Artificial Intelligence (ECAI 2020) / [ed] Giuseppe De Giacomo, Alejandro Catala, Bistra Dilkina, Michela Milano, Senén Barro, Alberto Bugarín, Jérôme Lang, IOS PRESS , 2020, Vol. 325, p. 8p. 1119-1126, article id FAIA200209Conference paper, Published paper (Refereed)
Abstract [en]

This paper presents an empirical study on the weights of neural networks, where we interpret each model as a point in a high-dimensional space – the neural weight space. To explore the complex structure of this space, we sample from a diverse selection of training variations (dataset, optimization procedure, architecture,etc.) of neural network classifiers, and train a large number of models to represent the weight space. Then, we use a machine learning approach for analyzing and extracting information from this space. Most centrally, we train a number of novel deep meta-classifiers withthe objective of classifying different properties of the training setup by identifying their footprints in the weight space. Thus, the meta-classifiers probe for patterns induced by hyper-parameters, so that we can quantify how much, where, and when these are encoded through the optimization process. This provides a novel and complementary view for explainable AI, and we show how meta-classifiers can reveal a great deal of information about the training setup and optimization, by only considering a small subset of randomly selected consecutive weights. To promote further research on the weight space, we release the neural weight space (NWS) dataset – a collection of 320K weightsnapshots from 16K individually trained deep neural networks.

Place, publisher, year, edition, pages
IOS PRESS , 2020. Vol. 325, p. 8p. 1119-1126, article id FAIA200209
Series
Frontiers in Artificial Intelligence and Applications, ISSN 1879-8314 ; 325
Keywords [en]
machine learning, deep learning, ai, computer vision
National Category
Computer Vision and Robotics (Autonomous Systems) Computer Sciences
Identifiers
URN: urn:nbn:se:liu:diva-169431DOI: 10.3233/FAIA200209ISI: 000650971301047ISBN: 9781643681016 (electronic)OAI: oai:DiVA.org:liu-169431DiVA, id: diva2:1467249
Conference
European Conference on Artificial Intelligence (ECAI 2020)
Note

Funding: Wallenberg Autonomous Systems and Software Program (WASP); strategic research environment ELLIIT

Available from: 2020-09-15 Created: 2020-09-15 Last updated: 2021-09-20

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full texthttp://ebooks.iospress.nl/volume/ecai-2020-24th-european-conference-on-artificial-intelligence

Authority records

Eilertsen, GabrielJönsson, DanielUnger, JonasYnnerman, Anders

Search in DiVA

By author/editor
Eilertsen, GabrielJönsson, DanielUnger, JonasYnnerman, Anders
By organisation
Media and Information TechnologyFaculty of Science & Engineering
Computer Vision and Robotics (Autonomous Systems)Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 296 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf