How I Tricked Claude into Leaking Your Deepest Secrets via AI Memory
I tricked Claude into leaking your deepest, darkest secrets
I discovered a critical flaw in Claude's memory system that allows attackers to silently exfiltrate personal data. By crafting a deceptive website that mimics a legitimate service, I tricked the AI into navigating a series of links that spelled out my name, employer, and hometown. This attack exploits the AI's web browsing capabilities to bypass security sandboxes, proving that our digital assistants can unknowingly leak sensitive information to malicious actors.
AI assistants like Claude have accumulated the most information-dense profiles on millions of people, becoming a high-fidelity reconstruction of you that could be used for blackmail, impersonation, or bypassing security questions.
- sonink
Its a bit wild to me that there hasnt been a pushback against enabling memories by frontier AI companies. This data is something advertisers could only dream off. Before AI, most of this data was approximated by whatever little information could be gleaned from the websites we visit. But now people are handing over their deepest darkest secrets and pretty much EVERYTHING to AI on a platter.
Maybe its just me who is paranoid because I happen to spend a fair bit of time in the advertising world, but the first thing I did when memory was launched on Claude/Chatgpt - was to switch them off. And it helps that they are not even useful, and would actually downgrade your experience by polluting the context of irrelevant details. I go one step ahead - if there is a personal discussion you want to have - maybe use another account like provided by the likes of companies like openrouter etc.
I would argue that we should have regulation that should prohibit the storage of user profile information by AI companies, and any such memories feature should exclusively reside on the users servers. Infact, maybe go one step ahead, that 'memory' firms cannot be owned by AI firms and vice versa.
- artisinal
Doesn’t surprise me.
Yesterday I learned that people run AI agents on their system with full admin rights. No containerisation or anything. Wild. Like we forgot 50 years of computer security overnight.
- port3000
My name in Claude is Silly Bean. I did it at first because it made me chuckle every time I opened Claude and it said 'Back again, Silly Bean?'
But turns out I was playing 4D cybersecurity chess
- adrian17
> After 15 minutes of confusion, it turned out Cloudflare had put a crazy robots.txt on my site without my consent (Cloudflare, love you guys, but this needs to stop).
Might be the first time I see someone complain about their website being protected from a scraper, instead of the other way around.
- rmunn
I've been running Claude Code in a VM, where I clone the GitHub repos I want it to work on (they're open source so no login info needed) but have no other credentials. I used to reset the VM every day, but that was getting to be a bit of a hassle so I switched to a monthly reset. But even so, it would be hard for Claude to exfil anything more than what open-source projects I've been working on in the past month (at worst). Which still could tell someone quite a lot about me, but most of that info is already out there available with a Google search — after all, when you contribute to open source projects, your name and email address get stored in immutable Git history.
But after seeing this, I think I might switch to a weekly VM reset rather than monthly.
BTW, if anyone is interested in a decent setup for an AI agent jail, the scripts at https://jai.scs.stanford.edu/arch-vm.html are what I used, plus adding a few more packages to the pacstrap command such as dotnet-sdk. I then made the guest root directory a BTRFS subvolume, so that I can snapshot it. Then spinning up a new VM is a `sudo btrfs subvol snap template-root newvm` command (basically instant) followed by running the `qemu-system-x86_64` command (takes a couple of seconds). It's easy, but I retain complete control over the contents of the VM. It's been great so far.
- NichoPaolucci
This is why I feel prompt injection is going to continue to be an issue. Fantastic that “Hi we are Cloudflare, give us your personal data” works.
Either we stunt the models to the point where they are not useful, or we allow things like this to seep in and create one of the most insecure concepts the internet (and maybe tech as a whole) has ever seen: a robot that can be tricked.
- bflesch
Creative use of social engineering, well done.
> "no bounty was awarded"
Ridiculous. Anthropic engineers are not just stupid to allow such a vuln in the first place, but they also try to hide such vulns from their bosses because a bounty payout would need to be explained to the finance team.
- hmokiguess
Tangential but I actually experienced recently something quite creepy and strange with Chat GPT iPhone app.
A close friend prompted it about some troubleshooting of a pet smart feeder and it responded with instructions but using my pet’s name to my friend.
I found that extremely strange for it to be a coincidence. My pet's name is not that generic for it to be in training data, and the connection to my friend makes it more strange to me.
That made me wonder if there’s cache pollution or some session data leakage in it exposing stuff. (My friend has been in our wifi for example)
Has anybody else noticed something like this?