liu.seSök publikationer i DiVA
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
Semantic Text Transmission via Prediction with Small Language Models: Cost-Similarity Trade-off
Indian Inst Technol Dharwad, India.
Indian Inst Technol Dharwad, India.
Indian Inst Technol Dharwad, India.
Linköpings universitet, Institutionen för datavetenskap, Databas och informationsteknik. Linköpings universitet, Tekniska fakulteten.ORCID-id: 0000-0003-4416-7702
2024 (Engelska)Ingår i: 2024 IEEE INTERNATIONAL CONFERENCE ON COMMUNICATIONS WORKSHOPS, ICC WORKSHOPS 2024, IEEE , 2024, s. 1244-1249Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

We consider the communication of natural language text from a source to a destination over noiseless and character-erasure channels. We exploit language's inherent correlations and predictability to constrain transmission costs by allowing the destination to predict or complete words with potential dissimilarity with the source text. Concretely, our objective is to obtain achievable ((c) over bar, (s) over bar) pairs, where (C) over bar is the average transmission cost at the source and (s) over bar is the average semantic similarity measured via cosine similarity between vector embedding of words at the source and those predicted/completed at the destination. We obtain ((c) over bar, (s) over bar) pairs for neural language and first-order Markov chain-based small language models (SLM) for prediction, using both a threshold policy that transmits a word if its cosine similarity with that predicted/completed at the destination is below a threshold, and a periodic policy, which transmits words after a specific interval and predicts/completes the words in between, at the destination. We adopt an SLM for word completion. We demonstrate that, when communication occurs over a noiseless channel, the threshold policy achieves a higher (s) over bar for a given (c) over bar than the periodic policy and that the (s) over bar achieved with the neural SLM is greater than or equal to that of the Markov chain-based algorithm for the same (c) over bar. The improved performance comes with a higher complexity in terms of time and computing requirements. However, when communication occurs over a character-erasure channel, all prediction algorithms and scheduling policies perform poorly. Furthermore, if character-level Huffman coding is used, the required (c) over bar to achieve a given <overline>s is reduced, but the above observations still apply.

Ort, förlag, år, upplaga, sidor
IEEE , 2024. s. 1244-1249
Serie
IEEE International Conference on Communications Workshops, ISSN 2164-7038, E-ISSN 2694-2941
Nationell ämneskategori
Telekommunikation
Identifikatorer
URN: urn:nbn:se:liu:diva-210352DOI: 10.1109/ICCWORKSHOPS59551.2024.10615608ISI: 001296276700207ISBN: 9798350304060 (tryckt)ISBN: 9798350304053 (digital)OAI: oai:DiVA.org:liu-210352DiVA, id: diva2:1920017
Konferens
59th Annual IEEE International Conference on Communications (IEEE ICC), Denver, CO, jun 09-13, 2024
Anmärkning

Funding Agencies|Swedish Research Council (VR), Excellence Center at Linkoping - Lund in Information Technology (ELLIIT); European Union [101096526]; European Union's Horizon Europe research and innovation programme under the Marie Sklodowska-Curie Grant [101131481]

Tillgänglig från: 2024-12-10 Skapad: 2024-12-10 Senast uppdaterad: 2024-12-10

Open Access i DiVA

Fulltext saknas i DiVA

Övriga länkar

Förlagets fulltext

Sök vidare i DiVA

Av författaren/redaktören
Pappas, Nikolaos
Av organisationen
Databas och informationsteknikTekniska fakulteten
Telekommunikation

Sök vidare utanför DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetricpoäng

doi
isbn
urn-nbn
Totalt: 34 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf