What Do Your Logits Know? (The Answer May Surprise You!)
Apple researchers just proved your 'private' model internals leak everything. Here's what attackers can extract.
Why it matters
Apple's research reveals that probing model internals can extract sensitive information thought to be inaccessible to users, raising critical security and privacy concerns for deployed vision-language models and exposing a gap in model owner threat models.
The key facts
10 to knowStudy focuses on information leakage through model internals (logits, residual streams)
First systematic comparison of information retention across representational levels
Vision-language models used as primary testbed
Identifies both unintentional and malicious information leakage vectors
Published by Apple Machine Learning Research
Challenges assumption that low-dimensional projections adequately protect model information
Research demonstrates systematic information leakage through logits and residual streams
Vision-language models used as testbed for probing model internals
Information retention across representational levels identified as security/privacy risk
Unintentional or malicious information access possible despite model owner assumptions
Go to the source
Apple Machine Learningmachinelearning.apple.com
Publisher excerpt: Recent work has shown that probing model internals can reveal a wealth of information not apparent from the model generations. This poses the risk of unintentional or malicious information leakage, where model users are able to learn information that the model owner assumed was inaccessible. Using…