Across the social sciences, many studies use cross-sectional designs that can show associations but are generally not able to support direct causal claims. Even so, authors of such papers may make or imply causal claims in their writing, which can blur the line between what the data can show and what the article appears to conclude.

To measure how often this happens, researchers analyzed 194,631 cross-sectional articles using large language models. Looking at the period from 1980 to 2024, they found that an average of 46% of articles contained causal language in their titles or abstracts. Since 2000, the annual rate has risen almost threefold, from 20% to 60%, suggesting that this wording is common enough to shape how readers encounter social science research at scale.

The researchers also examined the effects of this language in a human-subjects experiment involving 1,105 participants. They found that readers often said abstracts using this phrasing provided causal evidence. At the same time, methodological labels (β = −0.4, 95% confidence interval −0.56 to −0.19) and associational wording (β = −0.3, 95% confidence interval −0.43 to −0.07) reduced that tendency, underscoring how word choice can influence interpretation even when the underlying study design does not change.

In experiments with five LLMs, model summaries of these articles, with 100 examples for each model, sometimes amplified causal overstatement. In those summaries, the models removed hedges and introduced causal claims even when the original articles used strictly associational phrasing. Prompting the models to be cautious reduced this pattern, a finding with practical relevance as more readers encounter research through automated summaries rather than full papers.

The findings come from a recent paper by Calvin Isch, Timothy Dörr, Neil Fasching, Grace Jennings, and Duncan J. Watts. Isch is on the job market this year and works with Tetlock and Watts.