AI detectors are poor western blot classifiers: a study of accuracy and predictive values.
Details
Serval ID
serval:BIB_321918877FDA
Type
Article: article from journal or magazin.
Collection
Publications
Institution
Title
AI detectors are poor western blot classifiers: a study of accuracy and predictive values.
Journal
PeerJ
ISSN
2167-8359 (Electronic)
ISSN-L
2167-8359
Publication state
Published
Issued date
2025
Peer-reviewed
Oui
Volume
13
Pages
e18988
Language
english
Notes
Publication types: Journal Article
Publication Status: epublish
Publication Status: epublish
Abstract
The recent rise of generative artificial intelligence (AI) capable of creating scientific images presents a challenge in the fight against academic fraud. This study evaluates the efficacy of three free web-based AI detectors in identifying AI-generated images of western blots, which is a very common technique in biology. We tested these detectors on AI-generated western blot images (n = 48, created using ChatGPT 4) and on authentic western blots (n = 48, from articles published before the rise of generative AI). Each detector returned a very different sensitivity (Is It AI?: 0.9583; Hive Moderation: 0.1875; and Illuminarty: 0.7083) and specificity (Is It AI?: 0.5417; Hive Moderation: 0.8750; and Illuminarty: 0.4167), and the predicted positive predictive value (PPV) for each was low. This suggests significant challenges in confidently determining image authenticity based solely on the current free AI detectors. Reducing the size of western blots reduced the sensitivity, increased the specificity, and did not markedly affect the accuracy of the three detectors, and only slightly improved the PPV of one detector (Is It AI?). These findings highlight the risks of relying on generic, freely available detectors that lack sufficient reliability, and demonstrate the urgent need for more robust detectors that are specifically trained on scientific contents such as western blot images.
Keywords
Artificial Intelligence, Blotting, Western/methods, Blotting, Western/standards, Sensitivity and Specificity, Image Processing, Computer-Assisted/methods, Predictive Value of Tests, Humans, AI detection, Accuracy study, Fraud, Immunoblotting, Paper mills, Research ethics, Research integrity
Pubmed
Open Access
Yes
Create date
28/02/2025 12:14
Last modification date
01/03/2025 7:34