from Hacker News

Hackers can read private AI assistant chats even though they're encrypted

by pkilgore on 3/14/24, 1:10 PM with 10 comments

• by lolinder on 3/14/24, 1:50 PM

  The actual attack is described near the end of the article—ChatGPT and friends send tokens as fast as they are generated, and the encrypted data doesn't mask the size of the token. This allows the attacker to guess the size of each token as it comes in, which they feed into a specialized model that guesses the word sequence based on the token lengths.

  Mitigating this should be very simple—pad the token with null to some reasonable length before sending it, then clip it on the client side. This would result in slightly higher bandwidth usage, but not enough to be perceptible.

• by SushiHippie on 3/14/24, 2:04 PM

  Related article on frontpage:

  Cloudflare mitigates AI side channel attack | https://news.ycombinator.com/item?id=39703255

• by acureau on 3/14/24, 1:33 PM

  Interesting read, from the title I figured the article was going to be about "hackers" breaching accounts and reading past conversations. I don't think reading private chats is the way to put it, more like inferring topics of discussion with some accuracy

• by wkat4242 on 3/14/24, 3:41 PM

  Interesting attack, but it's easily mitigated. Just batch the responses into sentences or something instead of words. Or padding, as others have suggested. The services will adapt quickly and the issue will be solved.

  It also requires a full packet capture of the target making it not very easy to execute.

• by yunohn on 3/14/24, 1:55 PM

  This is actually a very interesting side channel attack for streaming data. They use an LLM to guess words from token length. Definitely, padding would help here.

  Nit: Title should say “guess” and not “read”.