liu.seSearch for publications in DiVA
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Temporal Predictive Coding for Gradient Compression in Distributed Learning
Linköping University, Department of Electrical Engineering, Communication Systems. Linköping University, Faculty of Science & Engineering.
Linköping University, Department of Electrical Engineering, Communication Systems. Linköping University, Faculty of Science & Engineering.ORCID iD: 0000-0001-5621-2860
Univ Paris Saclay, France.
KTH, Sweden.
2024 (English)In: 2024 60TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING, ALLERTON 2024, IEEE , 2024Conference paper, Published paper (Refereed)
Abstract [en]

This paper proposes a prediction-based gradient compression method for distributed learning with event-triggered communication. Our goal is to reduce the amount of information transmitted from the distributed agents to the parameter server by exploiting temporal correlation in the local gradients. We use a linear predictor that combines past gradients to form a prediction of the current gradient, with coefficients that are optimized by solving a least-square problem. In each iteration, every agent transmits the predictor coefficients to the server such that the predicted local gradient can be computed. The difference between the true local gradient and the predicted one, termed the prediction residual, is only transmitted when its norm is above some threshold. When this additional communication step is omitted, the server uses the prediction as the estimated gradient. This proposed design shows notable performance gains compared to existing methods in the literature, achieving convergence with reduced communication costs.

Place, publisher, year, edition, pages
IEEE , 2024.
Series
Annual Allerton Conference on Communication Control and Computing, ISSN 2474-0195, E-ISSN 2836-4503
Keywords [en]
Distributed learning; communication efficiency; predictive coding; event-triggered communication
National Category
Signal Processing
Identifiers
URN: urn:nbn:se:liu:diva-213915DOI: 10.1109/ALLERTON63246.2024.10735311ISI: 001444037100050Scopus ID: 2-s2.0-85211119410ISBN: 9798331541040 (print)ISBN: 9798331541033 (electronic)OAI: oai:DiVA.org:liu-213915DiVA, id: diva2:1962079
Conference
60th Allerton Conference on Communication Control and Computing, University of Illinois, Urbana, IL, sep 24-27, 2024
Available from: 2025-05-28 Created: 2025-05-28 Last updated: 2025-05-28

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Edin, AdrianChen, Zheng
By organisation
Communication SystemsFaculty of Science & Engineering
Signal Processing

Search outside of DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric score

doi
isbn
urn-nbn
Total: 33 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf