Text AI watermarks will always be trivial to remove
The EU AI Act will soon require AI outputs to be detectable as artificial, but text watermarking is fundamentally flawed. Unlike images, text can't be subtly altered without degrading quality. Google's SynthID embeds a statistical fingerprint in token choices, while OpenAI and Anthropic may use Unicode homoglyphs. However, both methods are easily stripped: homoglyphs can be replaced, and SynthID can be defeated by paraphrasing with any LLM. The Act's interoperability requirement further undermines security-by-obscurity, making text watermarks a compliance checkbox rather than a real safeguard.
Unlike image and video watermarks, text watermarks will always be trivial to remove.
- nonethewiser
Isnt this absurd? Say im brainstorming a resume bulet point. Its 15 words. I like it but want to condense it to a single line. I give it to the ai and tell it how much overflows and now it gives me back a simplified sentence and 11 words and some extra stenography constraint? What kind of rule could possibly not effect the quality of that output?
Ok say i do that on 30% of bullet points. Karen the hiring manager is vehemently anti AI. She gives my resume to her AI scanner and what does she find? This not a rhetorical question. Will it treat the text as a whole and not find it? Does it scan every combination of contiguous terms? It could scan bullet points but i could generate in pairs of 2. What about novels?
- firefoxd
I feel like this is going to end up being like cookie laws. It sounds good, I don't know how any one benefits from it.
Currently I can recognize AI text because I read thousands of ai generated text. I know that 110% of yahoo finance news is generated. I don't want to read an AI generated personal blog, but if I do what's the problem really? Other than the companies distinguishing AI text for getting better training data, how do people benefit from watermarked text exactly?
- simon84
The goal of the AI act is not to determine if an "oh yeah!" comment was AI generated. The target is long papers that falsely claim human review and can have real significant consequences.
E.g. research paper, law makers, lawyers, state policies, notaries,...
These are much longer content and thus statistically they will disclose a better guess at AI generated content.
Asking another AI to paraphrase will not erase the mark (which they are unaware about) but rather cumulatively add their own mark and make it easier to detect.
The problem is not to use AI, but to endorse the responsibility of the content you (as a human) deliver and somehow make sure that fake-news, biased content or unverified output is detected as early as possible.
- andy_xor_andrew
The article mentions you can always simply use a smaller, local, un-watermarked LLM to rephrase the original watermarked text. Which is true, sure.
But if we're talking about deterministically taking some watermarked LLM output and having a function removeWatermark(text), it won't necessarily be "trivial" to remove, because the watermark function itself need not be public. Only the API that tests for the watermark need be public, right?
Anthropic's magic watermark could be, like the article mentions, something like "every 7th semicolon has a N% chance to be a comma where N is the sum of the last X characters mod Y, and every character in the bit range q1...q2 has a Z% chance to..." etc etc etc. And if Anthropic controls those variables, it would be very difficult to determine the rule, even with some pretty advanced analysis (I would assume). And keep in mind, that example rule I mentioned is pretty naive, too. I expect the actual rule would be way more advanced and not so straightforward as "swap every <charX> for a <charY>"
- happytoexplain
Yeah, but it's better than nothing.
People underestimate the value of rules that only take malice and a little knowledge to break.
And they tend to exaggerate that underestimation if they... don't like the rule.
- Waterluvian
Maybe this is an information theory thing but can there exist a n“ I am human” shibboleth or is this concept fundamentally impossible?
It feel that today you can generally convince someone with text alone that you’re human. Beepity boopity zip zap zoopity today’s AIs aren’t this loose and derpy. Here’s a fTypo and my secret stash of dashes ——-–.
But one day even this won’t do, right?
- rsynnott
“Everyone will simply do fraud” - it is a bizarrely immoral world that the LLMs seem to have unleashed on us. Tech was kinda heading that way anyway, but our friends the magic robots really seem to have turbocharged it.
- defen
AI watermarks feel like they're approaching the problem from the wrong side - no matter what it will be possible to remove the watermark (Via manual rewriting, local LLLMs, etc). Instead it seems like we need "proof of human creation". And the only way I see that being possible is hardware-attested proof of keypresses. Which obviously has huge privacy implications, but how else would you actually know that a piece of content was produced by a human pressing keys on a keyboard? The proof would also need to include timestamps for the keypresses, so tell if someone is just copying from another window.