People fail to distinguish AI-written text from human writing

A new study set out to answer the question that haunts every content creator: can people really tell whether the text in front of them was written by a person or by a language model? The short answer is no. In blind tests, participants’ ability to separate AI-generated text from human-written text was almost indistinguishable from random guessing. The data leave little room for interpretation: if you tossed a coin, you would get similar results.

The surprising part of the findings is not only that people stumble, but who does a little better. One might assume that literary professionals—those whose job is creative writing—would be able to spot the difference between a human style and a machine-generated one. That assumption is wrong. The group that showed a slightly higher detection rate is the one that works with AI on a regular basis. The creators who publicly claim they can spot machine-written text from a distance are precisely the ones whose guesses are no more accurate than a coin toss.

Dina Weisberg, one of the researchers, offers an explanation. According to her, the data reveal a bias toward texts written by humans because people assume that creative writing requires uniquely human traits such as emotional understanding and life experience. That assumption, she says, leads people to underestimate AI’s actual capabilities. Public estimates of what machines can or cannot do, in her view, are outdated.

The study’s broader interpretation touches on something more basic. Most readers, it turns out, prefer clarity over literary sophistication. When the goal is comprehension, AI delivers the content without excessive cleverness. This leaves human writers with the pride of an inaccessible elitism but not the popularity. As the research notes with a hint of dry irony, those writers want both prestige and commercial success. In the realm of popularity, competition with machines that produce clear, on-point prose is becoming increasingly tough, and a shift in direction does not appear on the horizon.