Updated ,first published
School essays generated by Anthropic AI would be easily and reliably flagged thanks to an invisible watermarking system detailed by the company, although it is yet to describe exactly how it works.
As part of the system, Anthropic’s Claude AI would weave imperceptible signals throughout any generated text. Then, if someone suspects that AI-generated writing is being passed off as human, they could use an as-yet-undescribed detection mechanism to scan for those signals.
“Because the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing,” Anthropic said in a post.
“Watermarking will be applied at the model level, which means it will be present no matter which Claude product or surface the text comes from.”
The company has not described how the watermark works, beyond saying it’s part of the text, and that it won’t affect the meaning, quality, or readability of outputs. That’s led many observers to conclude that it’s similar to some existing text watermarking systems; the AI will reliably use certain patterns or combinations of words according to set rules. So as long as a detection system knows those rules, and assuming a long enough passage of text, it will confidently tell if Claude made it.
This detection system could be used on anything, from job applications to personal emails, but it could be most useful in schools. Teachers are increasingly being made to judge whether an essay or assignment has been written by students or generated by AI.
Anthropic’s commitment comes as part of its compliance with new European laws which demand AI companies build in reasonable transparency measures, though it said the watermarking would apply worldwide. Any new model introduced will include the watermarking from day one, while existing models will be updated with it in the future.
OpenAI has also agreed to the rules, and has developed its own text watermarking system, but has not yet rolled it out to ChatGPT publicly. Google has developed its own version, SynthID Text, which it applies to Gemini output. And all three companies, plus Apple, apply standard C2PA watermarks to generated images for detection, with Google adding its own additional watermark designed to survive cropping and re-formatting.
While teachers may be glad about the developments, watermarking of AI outputs is not without its critics.
Many writers have described detection systems as antidotes owned and sold by the same companies profiting from the poison, or as fire extinguishers designed by arsonists. Even before this latest round of systems, the AI detection industry has long been described by critics as a protection racket. There are tools that generate text, tools that detect generated text, and also tools that humanise generated text to avoid the detection. Subscription money is collected from users at every step.
Avid users of Claude have strongly rebuked the move, arguing it will punish regular users while covering for the worst and most dangerous use cases. On the ClaudeAI subreddit, users said watermarks will interfere with output quality, particularly for coding, will discourage the use of Claude to improve or check human-made work in case it inserts watermarks, and will be easily reverse-engineered by criminals who will make tools to strip the marks out. If their concerns are validated, that would mean someone who wrote a novel and only used AI for grammar checks could fail the detection tool and be blocked from publishing, while a foreign government propagandist could “prove” a completely AI-generated manifesto was legitimately human-written.
In its post, Anthropic does admit that using Claude to check human-made work may result in the work being flagged by the tool, meaning a positive result is only proof that the writing has been touched or processed by AI, not that it was fully written by it.
Conversely, a negative result does not mean human-written text, given the proliferation of other AI writing tools. So for schools, that would still mean maintaining a hard rule against all AI use, and deploying multiple detection suites.
Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.
Tim Biggs is a writer covering consumer technology, gadgets and video games.Connect via X or email.From our partners
About the Author
