It’s the latest tech challenge: identify when students, authors, and others depend on AI to produce content. Anthropic places invisible statistical watermarks in Claude-generated text. Google deploys a similar technique, SynthID, in Gemini. OpenAI is testing watermarking ChatGPT responses.
Under Europe’s Artificial Intelligence Act, new labeling rules took effect this summer. Companies must now mark AI-generated content and deepfakes that relate to public interest topics, including politics, public health, security, and the economy. The only exception is for content that has undergone genuine human review or editorial control.
While the US has no federal requirement to watermark AI-generated content, Congress has considered legislation, and several states have acted. California’s AI Transparency Act requires large providers to offer detection tools and embed origin disclosures in AI-generated images, video, and audio, though not text.
China goes further, requiring both visible and hidden labels for AI-generated text, images, audio, and video. South Korea also regulates AI-generated content but gives providers a choice over how to disclose it: deepfakes require clear labeling, while other AI-generated material can be marked through watermarks, pop-ups, or interface notices.
Whether the new rules succeed in reassuring readers remains unclear. Even the premise behind these requirements can be questioned — we don’t accuse anybody of cheating when they spellcheck their essays or use a calculator to do math. If AI can be anything from a glorified spellchecker to a ghostwriter, should it carry the same stigma? And will detection really reassure a public already uneasy about the technology?
But generative AI is different. It can contribute far more to the final work than a calculator, suggesting arguments, organizing ideas, rewriting passages, or producing entire sections of text. Most uses fall somewhere between those extremes, and the desire for transparency runs deep. Polish Nobel Prize-winning novelist Olga Tokarczuk faced a massive backlash after she acknowledged recently asking an AI chatbot, “Darling, how could we develop this beautifully?” Although she later clarified that she only employs AI for research, not to write her books, her experience suggests that, even as AI becomes commonplace, its use still carries a reputational cost.
Here’s how watermarking works. AI models follow a hidden pattern when choosing their words. If “start” and “begin” would both fit naturally, for example, the system might be designed to prefer one of them. Across a long enough passage, those small choices add up to a pattern that software can detect, allowing it to judge whether AI generated the text.
Watermarking does not reduce the quality of AI-generated texts. Google tested SynthID-Text across almost 20 million Gemini responses and found that watermarked answers rated just as highly as answers produced without the watermark. Anthropic says the changes in word choice are subtle enough that a reader will be unable to tell that the text has been watermarked.
But the technique represents no panacea. It can show only that a large language model was probably involved. It cannot distinguish whether AI edited, wrote, or simply checked the grammar of the text. Take the ordinary problem of composing in English. The British Council estimates that people who speak English as an additional language outnumber native speakers by about four to one. In multinational companies and international organizations, English is often the working language between people who grew up speaking something else.
Generative AI helps improve writing quality. A Carnegie Mellon study of graduate students found that AI cut average writing time by 65% and improved writing quality, with larger gains among students using English as a second language. A non-native English-speaking analyst can spend a week researching a subject, building the argument and checking the evidence, then ask AI to make the prose sound natural. The model may end up choosing every word in the final version. A watermark can flag the machine’s role in producing the text, but it cannot show who did the thinking behind it.
The real problem comes when watermarks convict people of cheating. An employer could conclude that an employee did not write a report. A publisher could question whether an author actually wrote a submitted article or manuscript. A recruiter could discount a cover letter or writing sample. In each case, the watermark would show only that AI touched the text, not how much of the work the human did.
Admittedly, universities face a genuine AI cheating challenge. University of Reading researchers found that 94% of fully AI-written exam answers submitted into real assessments escaped detection. AI can clearly be used to outsource the work universities are trying to assess. But a watermark cannot distinguish that from a student who wrote an essay and one who asked an AI to improve the English.
Good reasons to watermark AI-generated media exist. A fake recording of a politician, a cloned voice authorizing a transfer, or a fabricated photograph can mislead because people believe the underlying event happened.
But watermarking will fail to solve the challenge of coming to terms with AI content. People dictate, translate, prompt, edit, fact-check, and rewrite, often several times and with several tools. Watermarks may simply reveal that the writer needed help with English.
Dr. Anda Bologa is a Senior Researcher with the Tech Policy Program at the Center for European Policy Analysis (CEPA).
Bandwidth is CEPA’s online journal dedicated to advancing transatlantic cooperation on tech policy. All opinions expressed on Bandwidth are those of the author alone and may not represent those of the institutions they represent or the Center for European Policy Analysis. CEPA maintains a strict intellectual independence policy across all its projects and publications.
Tech 2030
A Roadmap for Europe-US Tech Cooperation
