liu.seSök publikationer i DiVA
Ändra sökning
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
From Privacy Chains to ChainShield: Structured Privacy Risks and Defense in Vision-Language Models
Linköpings universitet, Institutionen för datavetenskap, Databas och informationsteknik. Linköpings universitet, Tekniska fakulteten.ORCID-id: 0009-0001-0554-3720
Linköpings universitet, Institutionen för datavetenskap, Databas och informationsteknik. Linköpings universitet, Tekniska fakulteten.ORCID-id: 0000-0003-2391-5951
Linköpings universitet, Institutionen för datavetenskap, Databas och informationsteknik. Linköpings universitet, Tekniska fakulteten.ORCID-id: 0000-0003-1367-1594
2025 (Engelska)Ingår i: PROCEEDINGS OF THE 24TH WORKSHOP ON PRIVACY IN THE ELECTRONIC SOCIETY, WPES 2025, Association for Computing Machinery (ACM), 2025, s. 116-133Konferensbidrag, Publicerat paper (Refereegranskat)
Abstract [en]

Vision-Language Models (VLMs) are increasingly deployed in applications that interpret and generate information from visual and textual inputs. While powerful, these models pose emerging privacy risks. In this paper, we introduce the concept of privacy chains: structured narratives that emerge when adversaries aggregate outputs from VLMs across multiple images, often exposing sensitive information even when the individual outputs are seemingly innocuous. Using LangChain, an open-source orchestration framework, we show how identity-linked data extracted via both benign and targeted prompts can be compiled into detailed timelines of private behavior, significantly amplifying privacy threats. To systematically assess this risk, we develop a privacy leakage pipeline within the Visual Question Answering (VQA) framework and evaluate six open-source VLMs across three tailored datasets: Celebrity, Car, and Tattoo. Our analysis reveals substantial and model-dependent privacy leakage, even from general-purpose queries. To mitigate this threat, we propose ChainShield, a white-box adversarial defense that applies targeted, imperceptible perturbations to images. ChainShield reduces privacy-relevant outputs by redirecting VLM responses toward benign alternatives, while preserving image realism. Our experiments show that ChainShield substantially lowers privacy leakage across models and datasets, effectively disrupting the formation of privacy chains.

Ort, förlag, år, upplaga, sidor
Association for Computing Machinery (ACM), 2025. s. 116-133
Nyckelord [en]
Privacy; Vision-Language Models; LangChain; Adversarial Attacks
Nationell ämneskategori
Bioinformatik (beräkningsbiologi)
Identifikatorer
URN: urn:nbn:se:liu:diva-220968DOI: 10.1145/3733802.3764048ISI: 001656324800010Scopus ID: 2-s2.0-105023662249ISBN: 9798400718984 (tryckt)OAI: oai:DiVA.org:liu-220968DiVA, id: diva2:2034558
Konferens
24th Workshop on Privacy in the Electronic Society-WPES, Taipei, TAIWAN, oct 13, 2025
Anmärkning

Funding Agencies|Swedish Research Council (VR); Graduate School in Computer Science (CUGS) at Linkoping University

Tillgänglig från: 2026-02-02 Skapad: 2026-02-02 Senast uppdaterad: 2026-06-17

Open Access i DiVA

Fulltext saknas i DiVA

Övriga länkar

Förlagets fulltextScopus

Person

Le, Minh-Ha

Sök vidare i DiVA

Av författaren/redaktören
Liu, MinxingLe, Minh-HaCarlsson, Niklas
Av organisationen
Databas och informationsteknikTekniska fakulteten
Bioinformatik (beräkningsbiologi)

Sök vidare utanför DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetricpoäng

doi
isbn
urn-nbn
Totalt: 26 träffar
RefereraExporteraLänk till posten
Permanent länk

Direktlänk
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Annat format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annat språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf