AI-Written English Uses More Constructions Linked to Reading Difficulty Than Human Writing, Study of 14 Models Finds
Researchers at New York University and the University of Texas at Austin measured markers of reading difficulty in AI-generated English. Their experiments also suggest that training models to keep answers short can reshape their prose style.
On October 21, researchers at New York University and the University of Texas at Austin released a paper analyzing how often markers of reading difficulty show up in English sentences written by AI. The team compared text from 14 language models against human writing. Several of the models, they found, used constructions linked to reading difficulty more frequently than people did.
Which constructions do AI models favor?
The researchers gathered model-generated text from conversations, coding tasks, and essay writing. They used two points of comparison: writing produced by humans, and English sentences that had been translated into Japanese and then back into English. The features they measured included rare word pairings, long distances between words that grammatically belong together, and stacked noun constructions.
In the conversational data, all nine of the top-tier and near-top-tier models exhibited two traits more often than humans. The first was joining words that do not normally appear together. The second was placing an inanimate object in the subject position and attaching a verb that describes a human action.
Many of the models also showed a higher rate of grammatically linked words being separated by long stretches of text, as well as stacked nouns. The degree varied from model to model, however. Given the same question, how complex the resulting sentences were depended on which model answered.
Does training models to give shorter answers make their writing easier to read?
The team also tested whether response length affects style. They continued training Qwen2.5-1.5B-Instruct while penalizing long answers. The stronger the penalty, the more the model gravitated toward phrasings that packed the same meaning into fewer words. The same tendency appeared in some constructions that were not used in the training.
The researchers explain that training a model to compress meaning can shift its style. To account for the writing style of the commercial models they surveyed, though, follow-up work would be needed to confirm whether the same mechanism was at play in their actual training.
Are newer versions easier to read?
Some models scored lower on the reading-difficulty measures than their predecessors. In the paper, Claude Opus 5.5 posted lower scores than Opus 5 on 19 of 21 comparison items. The researchers also report substantial differences between model families.
The study was limited to English, a restricted set of conversation topics, and code repositories. Korean sentences were not analyzed. Whether the same features appear in Korean-language responses, and how much they actually affect readers’ comprehension, remains unverified.