Hackers poison ChatGPT and Gemini to hand users fake support numbers for Delta, Chase, and hundreds more
Hackers influence ChatGPT and Gemini to direct users to scam centers
Attackers are flooding the web with GEO-optimized posts, PDFs, and fake support pages to trick AI assistants into citing fraudulent phone numbers, emails, and login pages. A detection system found 374 companies affected across ChatGPT, Gemini, and Google AI Overview, including Delta, Lufthansa, Chase, and Bank of America. The automated campaigns outpace takedowns, and Google and OpenAI dismissed the reports as out of scope or not reproducible.
AI cannot distinguish between truth and lies.
- pluc
That's happening across the board when it comes to models that feed off online information. Create a website with a false claim, have AI slurp it up and spit it back out when a user asks a question.
Happy elections.
- mbauman
I've been seeing the back-side of this as a mod at the Julia Discourse. Pagerank/SEO spam is ridiculously obvious and trivial to detect/block — links are all that matter. Many GEO spam posts are similarly obvious: someone posting about a cryptocurrency customer support hotline on a programming language forum is definitely spam. But recently spammers/scammers been using AI to _tailor_ posts to look much more authentic, posting "How to use Julia to analyze market statistics" referencing (without links!) a particular crypto exchange and then (sometimes) going back later to edit in phone numbers or the like.
What makes this even more painful is seeing all the good faith answers that such GEO posts spur from the community. It's far more abusive than SEO spam.
- QuantumNoodle
You don’t need to be a hacker to do this. My elderly neighbor searched for some version of “Microsoft help center phone number,” but Gemini suggested a scam number based on her search terms. She was scammed so much that I needed to spent the weekend helping her wipe computers, lock down the digital accounts and deal with identity theft.
- neya
Imagine working decades and decades on PageRank™ only to throw it all away because you felt you've been left behind in the AI race. This is how you lose focus.
- jimbo456
AI providers like ChatGPT and Gemini should be held criminally liable for instances where their products enabled a scam