liu.seSearch for publications in DiVA
Endre søk
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf
From Privacy Chains to ChainShield: Structured Privacy Risks and Defense in Vision-Language Models
Linköpings universitet, Institutionen för datavetenskap, Databas och informationsteknik. Linköpings universitet, Tekniska fakulteten.ORCID-id: 0009-0001-0554-3720
Linköpings universitet, Institutionen för datavetenskap, Databas och informationsteknik. Linköpings universitet, Tekniska fakulteten.ORCID-id: 0000-0003-2391-5951
Linköpings universitet, Institutionen för datavetenskap, Databas och informationsteknik. Linköpings universitet, Tekniska fakulteten.ORCID-id: 0000-0003-1367-1594
2025 (engelsk)Inngår i: PROCEEDINGS OF THE 24TH WORKSHOP ON PRIVACY IN THE ELECTRONIC SOCIETY, WPES 2025, Association for Computing Machinery (ACM), 2025, s. 116-133Konferansepaper, Publicerat paper (Fagfellevurdert)
Abstract [en]

Vision-Language Models (VLMs) are increasingly deployed in applications that interpret and generate information from visual and textual inputs. While powerful, these models pose emerging privacy risks. In this paper, we introduce the concept of privacy chains: structured narratives that emerge when adversaries aggregate outputs from VLMs across multiple images, often exposing sensitive information even when the individual outputs are seemingly innocuous. Using LangChain, an open-source orchestration framework, we show how identity-linked data extracted via both benign and targeted prompts can be compiled into detailed timelines of private behavior, significantly amplifying privacy threats. To systematically assess this risk, we develop a privacy leakage pipeline within the Visual Question Answering (VQA) framework and evaluate six open-source VLMs across three tailored datasets: Celebrity, Car, and Tattoo. Our analysis reveals substantial and model-dependent privacy leakage, even from general-purpose queries. To mitigate this threat, we propose ChainShield, a white-box adversarial defense that applies targeted, imperceptible perturbations to images. ChainShield reduces privacy-relevant outputs by redirecting VLM responses toward benign alternatives, while preserving image realism. Our experiments show that ChainShield substantially lowers privacy leakage across models and datasets, effectively disrupting the formation of privacy chains.

sted, utgiver, år, opplag, sider
Association for Computing Machinery (ACM), 2025. s. 116-133
Emneord [en]
Privacy; Vision-Language Models; LangChain; Adversarial Attacks
HSV kategori
Identifikatorer
URN: urn:nbn:se:liu:diva-220968DOI: 10.1145/3733802.3764048ISI: 001656324800010Scopus ID: 2-s2.0-105023662249ISBN: 9798400718984 (tryckt)OAI: oai:DiVA.org:liu-220968DiVA, id: diva2:2034558
Konferanse
24th Workshop on Privacy in the Electronic Society-WPES, Taipei, TAIWAN, oct 13, 2025
Merknad

Funding Agencies|Swedish Research Council (VR); Graduate School in Computer Science (CUGS) at Linkoping University

Tilgjengelig fra: 2026-02-02 Laget: 2026-02-02 Sist oppdatert: 2026-06-17

Open Access i DiVA

Fulltekst mangler i DiVA

Andre lenker

Forlagets fulltekstScopus

Person

Le, Minh-Ha

Søk i DiVA

Av forfatter/redaktør
Liu, MinxingLe, Minh-HaCarlsson, Niklas
Av organisasjonen

Søk utenfor DiVA

GoogleGoogle Scholar

doi
isbn
urn-nbn

Altmetric

doi
isbn
urn-nbn
Totalt: 26 treff
RefereraExporteraLink to record
Permanent link

Direct link
Referera
Referensformat
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Annet format
Fler format
Språk
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Annet språk
Fler språk
Utmatningsformat
  • html
  • text
  • asciidoc
  • rtf