Doc Searls Weblog

How Claude marks AI-generated content is the most self-destructive utterance I’ve ever heard coming out of an AI company. Wow:

When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response.

Because the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing. Watermarking will be applied at the model level, which means it will be present no matter which Claude product or surface the text comes from.

Will any writer, of anything, ever want to use any Claude text anywhere now? I’ve been a satisfied Claude customer, and I’m ready to bail completely. And I’m not alone. Because watermarks in text are invisible time bombs ready to scream, “The author didn’t write this! I did!” when some reader or their machine uses the right probe.

I feel bad for all the writers whose work depends on AI composition. For example, Daniel Barkhuff. In Hey Guys, I use AI! And so does everyone, he writes,

My process isn’t complicated. I sit down and write about a page, maybe 450 words. It’s not elegant. It’s not structured. It’s basically a brain dump. Half sentences, ideas that don’t quite connect yet, things I’d say out loud but that look ridiculous when you see them on the screen. It’s intellectual vomiting. A rough sketch of what I’m trying to say.

Then I paste it into AI and say something like, “Hey, give me an essay.”

And it does. (The tell is a paragraph, then a skipped line, then one sentence, preferably short and under 6 words).

I wrote the parenthetical part of the above.

But here’s the thing people misunderstand about that step. The machine isn’t doing the thinking. It isn’t generating the ethics, the argument, or the moral framework. All of that was already there in the messy words I wrote first. What the machine is doing is what editors have always done: taking raw thought and making it readable. AI doesn’t think. It formats.

Wrong on three counts. 

1. Editors edit the work of other writers. They don’t write for other writers. If they do, it’s called ghostwriting.
2. AIs don’t think, any more than they aren’t trained, don’t have information, and don’t learn. They emulate all those verbs, extremely well. That’s why—
3.  It sells AI very short to say it just “formats.”

With that post, Dr. Barkhuff watermarked everything he has written since, and I’ve read him less for it. And I hate to say that because I respect and admire him a great deal. And I’m sure he’s actually a good writer. 

While I understand why the EU requires this kind of shit, it throws monkey wrenches all over the place—because ghostwriting isn’t the only issue here.

John Gruber says,

Anthropic’s (original) support document unambiguously claims that’s what their system will enable. So if that were true, I couldn’t see what was left other than hiding invisible characters within the text.

My error was believing Anthropic that their system wouldn’t adulterate and corrupt the semantics of the text their models generate. That is in fact exactly what they plan to do…

The idea that anything other than my needs should factor into the generation of text for me is patently offensive.

This isn’t just about text one might generate with the intention of passing it off as their own natural work. This isn’t even about LLM proofreading of work written by hand. Anthropic is saying that all new Claude models are going to adulterate every single bit of text longer than 200 tokens (~150 words) they generate, including everything it presents to its users to read. So even in a private conversation between a user and Claude, which will never be read by anyone other than the user, Claude will begin making word choices in the name of marking its output in statistically predictable ways rather than maximizing clarity and precision.

Jeff Jarvis:

I’m angry as a writer.

Anthropic’s method will alter its models’ word choices in a pattern based on a key only it knows, allowing it to detect writing (or editing or translation) by its machine — to an uncertain degree of certainty. The company contends there is no difference in meaning between the otherwise random choices “grey” or “overcast” in, for example, the context of weather. So what’s the issue?

Thus Anthropic declares words fungible, language random, choice meaningless. Not to me. I try to select my words as carefully as I can for a number of considerations: style, tone, rhythm, avoiding repetition, but most of all meaning. You may quibble with my choices, but they are mine, not yours; that’s what makes my writing mine, to communicate what I wish to communicate. “Gray” and “overcast” are not the same, damnit. That’s the reason we have both of them.

So I resent Anthropic et al dismissing the value of a writer’s selection of one word over another for the impact that decision has on expression, on impression — occasionally on art.

We humans might not be able to detect whether the machine or the key is being used. That’s not the point. The selections/changes/alterations made in accordance with the key will affect sense and style on an ever-larger proportion of text entering discourse, and the fact that Anthropic (and the EU) consider this acceptable, unimportant, even trivial has cultural impact on the worth of words. They devalue writing. At the very moment when LLMs commodify writing and writers, this is just another kick to the kidneys for a challenged craft.

Kieth Teare translates Anthropic this way: “My product is so bad for mankind that I plan to place an indelible fingerprint that I created it so that everybody can know I was there.”

I don’t subscribe to Ben Thompson’s Stratechery, so I can’t find a passage to quote, so here’s Keith:

Watermarking is Nuts

Ben Thompson’s Stratechery piece belongs at the top of this week’s issue because he nails this truth.

A watermark starts from suspicion. It assumes AI involvement is the important fact, and that the task of institutions is to detect it after the fact. Thompson’s objection is simple and right: a mark may only show that Claude proofread, translated, summarized, or converted human-origin work. A missing mark does not prove AI was absent. It can accuse legitimate human work and miss machine-written work at the same time. I read a piece by my good friend Saul Klein this week and Substack’s partner said it was 100% AI written. It isn’t. Sure Saul probably used AI, but his ideas are solid and clear in the piece. They are not those of the tool he used.

More importantly, asking if AI has been used is the wrong moral question. We do not mark work because a calculator helped with the arithmetic, a compiler helped with the code, a camera helped with the image, or a search engine helped with the facts. We judge the result and the responsibility behind it. Is it true? Is it useful? Is it accountable? Is the author using the tool honestly? The bad acts are fraud, fake sources, fake evidence, fake authorship, hidden manipulation, and unaccountable slop. The bad act is not using AI. The vast majority of AI is used in these “good” ways, not in the “bad” ones.

I hope OpenAI, Google, and the other operators on the great AI frontier see this as a perfect opportunity to stand out in the marketplace by not watermarking text.