“Peer reviewers cannot pick up every mistake in scientific papers, so I think we need to look for different solutions that can help us in increasing the quality and robustness of scientific studies,” she said. “A.I. could definitely play a role in that.”
Renee Hoch, manager of the publication ethics team at the Public Library of Science, or PLOS, which like Frontiers is an open-access publisher, said her organization also used software tools to detect potential conflicts of interest between authors and editors, but not reviewers. Instead, referees are asked to self-report problems, and action is taken on a case-by-case basis.
Dr. Hoch, however, said that an A.I. tool like AIRA that highlights a reviewer’s potential conflicts would be useful in relieving some of the burden associated with manually conducting these checks.
Springer Nature, the world’s second-biggest scholarly publisher, is also developing A.I. tools and services to inform peer review, said Henning Schoenenberger, the company’s director of product data and metadata management.
Despite the rise of A.I. tools like statcheck and AIRA, Dr. Nuijten emphasized the importance of the human role, and said she worried about what would happen if technology led to the rejection of a paper “out of hand without really checking what’s going on.”
Jonathan D. Wren, a bioinformatician at the Oklahoma Medical Research Foundation, echoed that sentiment, adding that just because two researchers had previously been co-authors on a paper didn’t necessarily mean they couldn’t judge each other’s work objectively. The question, he said, is this: “What kind of benefits would they have for not giving an objective peer review today — would they stand to gain in any sort of way?”
That’s harder to answer using an algorithm.
“There’s no real solution,” said Kaleem Siddiqi, a computer scientist at McGill University in Montreal and the field chief editor of a Frontiers journal on computer science. Conflicts of interest can be subjective and often difficult to unveil. Researchers who have often crossed paths can be most suitable to judge each other’s work, especially in smaller fields.