0
0
A new study has estimated that at least 13.5% of biomedical research abstracts published in 2024 were processed with the help of large language models (LLMs), marking a significant shift in the style and vocabulary of scientific writing in the field. The analysis, conducted by researchers from the University of Tübingen, Germany, examined more than 15 million biomedical papers published between 2010 and 2024 and was published in the journal Science.
The researchers observed that the emergence of AI-powered LLMs—systems trained on vast amounts of text to generate human-like language—has led to a notable increase in the use of certain “stylistic words” such as delves, showcasing, underscores, potential, findings, and critical. These words, favored by AI systems, are typically verbs and adjectives that affect the style rather than the content of abstracts.
To arrive at their findings, the team employed a novel “excess word” framework, inspired by public health methods used to estimate excess deaths during the COVID-19 pandemic. By comparing word usage patterns before and after the widespread adoption of LLMs, the researchers were able to estimate the proportion of abstracts likely influenced by AI.
The impact of AI use varied by field and geography:
-
In computational biomedical research, the figure rose to about 20%, possibly reflecting greater familiarity and willingness to adopt new technologies among computer scientists.
-
In some disciplines, countries, and journals, the proportion reached as high as 40%.
-
In non-English speaking countries, LLMs are often used to assist with English language editing, further increasing their prevalence.
The authors noted that the shift in vocabulary brought about by AI has been “unprecedented,” even surpassing the effect of major world events such as the COVID-19 pandemic on scientific writing styles. However, they cautioned that factors such as shorter publication timelines in computational fields may have contributed to earlier detection of AI use, and that the results should be re-evaluated after additional publication cycles.
“We study vocabulary changes in more than 15 million biomedical abstracts from 2010 to 2024 indexed by PubMed and show how the appearance of large language models led to an abrupt increase in the frequency of certain style words,” the authors wrote.
Disclaimer:
This article is based on the findings of a single study, which used indirect linguistic analysis to estimate AI involvement in biomedical abstracts. The actual proportion of AI-assisted writing may vary, and further research is needed to validate these results across different fields and over time. The findings do not imply that AI-generated content replaced scientific rigor or peer review in these publications.
