Substack has identified a real problem. It may have chosen the worst possible solution.
On July 21, Substack co‑founder Chris Best announced a new feature called “scan for AI text.” Readers can run posts, Notes, comments, and replies longer than 100 words through an AI detector made by Pangram. The resulting report estimates how much of the text was written by a human and how much was generated or assisted by artificial intelligence.
An unexplained algorithmic score is not proof of authorship.
Best calls the problem “Claudefishing”: Passing off machine‑generated writing as the product of a human mind. He argues that readers deserve to know whether they are engaging with a person or, in his words, something “written by no one.”
That concern is legitimate. Anyone who spends much time online can recognize the rise of AI slop: The smooth, bloated, strangely lifeless prose that says very little in far too many words. It’s taking over LinkedIn, search results, marketing blogs, and, increasingly, journalism.
But Substack’s solution risks creating another problem: a culture in which every writer is treated as a potential fraud and every reader is encouraged to become an investigator.
Pangram is good, but still not good enough
Pangram is not one of the primitive AI detectors that declares that Dickens used ChatGPT. It may be the most capable product in its category.
The system is a machine‑learning classifier trained on both human‑written and AI‑generated documents. It breaks text into tokens, converts them into numerical representations, and looks for patterns its model associates with human or machine authorship.
Pangram says it repeatedly searches large collections of human writing for passages its model gets wrong, adds those difficult examples to its training data, and retrains the detector.
Its current model, Pangram 3.3, also uses an architecture called EditLens, which attempts to estimate not merely whether AI was involved but how extensively a human text may have been edited by AI.
Pangram claims an overall accuracy above 99% and a false‑positive rate of one in 10,000 human documents. Independent research has generally found it far more accurate than competing detectors. One University of Chicago study found almost no false positives among roughly 3,000 medium‑ and long‑form samples.
That sounds reassuring until Pangram is applied to individual human beings.
A detector can be excellent at identifying broad trends across millions of documents and still be inappropriate as a lie detector for a particular writer. At sufficient scale, even a one‑in‑10,000 error rate produces innocent people who must somehow prove that they wrote their own words.
Pangram CEO Max Spero has said the detector should “never be the ending arbiter.” But he has also acknowledged that the model’s internal reasoning is largely uninterpretable. It can provide a score, but it cannot necessarily explain which meaningful piece of evidence caused it to reach its verdict, according to the Atlantic.
That is a serious problem when the verdict carries the implicit accusation that an author is a liar.
Substack’s own moral authority was falsely flagged
The irony is hard to miss.
In his announcement, Best invokes writer Freddie deBoer to explain why undisclosed AI writing is a betrayal of the reader. Best quotes from a June essay by deBoer arguing that the value of art lies partly in knowing there is a human consciousness behind it.
But just two days before Best announced the Pangram integration, deBoer published a detailed account of Pangram failing to recognize his own human writing.
RELATED: Is Google's next Android feature always-on eavesdropping?
Yuliia Kaveshnikova/Getty Images
Someone had accused deBoer of using AI after Pangram classified a roughly 300‑word section of one of his essays as entirely AI‑written. DeBoer knew that was false.
When he submitted the full 5,000‑word essay, however, Pangram declared it entirely human. When he divided the supposedly artificial passage into smaller pieces, those pieces were also classified as human.
The same words became human or artificial depending on how much surrounding text deBoer submitted.
So the writer Substack chose as its moral authority on the importance of human authorship had just become a case study in the inability of Substack’s chosen detector to identify human authorship reliably in an individual case.
Best apparently agrees with deBoer that readers should not be deceived about whether a human being stands behind a piece of writing. But Pangram had placed deBoer in exactly the opposite position: A human writer was forced to defend himself against a machine suggesting that no human stood behind his words.
That contradiction should have given Substack pause. Instead, it gave every reader access to the machine.
Technology journalist Taylor Lorenz was also accused of using AI after Pangram flagged her work. Spero later investigated and concluded that the detector had made a mistake. Lorenz was fortunate enough to have an edit history supporting her account.
How many ordinary writers maintain a forensic record of every keystroke?
Meanwhile, determined AI users can still fool detectors. Researchers have shown that paraphrasing systems and even text generated by certain base models can appear human to Pangram and other commercial products.
The detector may therefore place the greatest burden on honest writers, while sophisticated slop merchants adapt their methods and move on.
A human writer trapped by a machine
I’ve already seen what detector paranoia does to people.
A local author I know spent months rewriting her book because she could not persuade an AI detector to score it below roughly 30% AI‑generated.
She is an older woman who has written numerous books. She is not immersed in technology and had no secret scheme to automate her writing. Yet the detector’s judgment became more credible to her than her own knowledge of how the book had been created.
So she rewrote it. Then she rewrote it again.
She changed genuine human sentences because a machine found them insufficiently human. Months of work were swallowed by an attempt to satisfy an algorithm that could neither know her nor explain itself to her.
I don’t know whether Pangram would have produced the same result, but that’s beside the point. The tragedy is that we have taught writers to regard an unexplained percentage produced by a machine as a moral verdict.
Substack is now placing that dynamic directly inside a platform supposedly built around the relationship between writers and readers.
The detector becomes the editor
Substack has included some safeguards.
The scan is not automatically displayed on every post. A reader must request it. Writers can add a “how I make this” statement explaining their process, report an apparent error, or disable detection on their work.
But Substack’s support documentation says that disabling it causes readers to see an “AI detection unavailable” notice.
That may be technically neutral, but anyone familiar with online pile‑ons knows how it will be interpreted: What is this writer hiding?
And what happens after a false result begins circulating?
RELATED: Private companies are tracking your location without notice or a warrant — and it's all perfectly legal
Moor Studio/Getty Images
Substack’s published procedure offers a feedback form and the ability to remove a scan. It does not offer anything resembling due process.
Pangram can correct its model later, but it cannot pull back screenshots, repair a damaged reputation, or stop a suspicious reader from repeating the accusation elsewhere.
The company may investigate complaints in good faith. Its engineers may be responsible people doing impressive work. But the writer is still being judged by a proprietary system whose reasoning neither the writer nor the reader can inspect.
There is also a deeper irony here. Substack is installing an artificial intelligence system to judge whether human beings are writing like artificial intelligence.
Suppose a writer scans a draft and Pangram says it is 30% AI‑assisted. The writer changes the offending passages until the score falls to zero.
Has he made the writing more human? Or has he allowed an AI system to edit his work through negative reinforcement?
The detector may not have supplied any words, but it has shaped the final text. The writer is no longer composing solely for readers. He is writing to please the machine that certifies his humanity.
By Substack’s own standards, does that count as AI‑assisted content?
The incentives are already changing. Independent scholar and entrepreneur Justin Murphy recently announced that he is co-founding a startup designed to help people protect themselves from AI detectors. At first glance, that sounds troubling. On second thought, it may be inevitable.
Once platforms begin judging writers with AI, someone will build AI to defeat those judges. Then the detectors will improve. Then the evasion tools will improve. We will have created an arms race in which one artificial intelligence writes, another disguises the writing, and a third attempts to expose it. Meanwhile, the actual question readers care about — whether the work contains original, thoughtful, compelling ideas — gets pushed farther into the background.
Judge the work
Personally, I don’t care much whether a writer used AI somewhere in the process.
There is nothing to be gained by staring suspiciously at every sentence and wondering whether a chatbot touched it. That road ends in paranoia, accusations, and writers deliberately making their prose worse because polished writing has become evidence of guilt.
I care whether the author has an original and compelling idea. I care whether it is presented coherently and clearly. I care whether the writer has done the work necessary to understand the subject and whether he is willing to stand behind what he publishes.
I also hate AI slop. Its laziness offends me.
But slop is not objectionable because it was made by a machine. It is objectionable because it is derivative, padded, evasive, and boring.
Humans produced plenty of slop before ChatGPT. Artificial intelligence has merely made it cheaper and faster. Arguably, most AI slop is better than the sort of slop produced by cut-rate third-world blog writers.
Editors should reject bad work. Readers should unsubscribe from writers who waste their time. Publications may reasonably require writers to disclose how they use AI. Serious accusations may sometimes justify examining drafts, notes, sources, and document histories.
But an unexplained algorithmic score is not proof of authorship.
Substack says it wants to preserve trust between writers and readers. Trust, however, cannot be manufactured by inviting readers to run secret authenticity tests on every post.
A platform built around direct relationships should not insert a machine informant between the writer and the reader.
