Put the whole issue of the paper mills aside for a minute. Here's how a normal process works:

The author(s) of a study send it in to a relevant journal in the relevant field.

The study is then perused by the journal editor or assistants (some of whom may be graduate students!) to determine possible peer reviewers. Generally speaking, the journal editor will be unpaid, or have a small stipend, but their main job is as a professor in some institution. The graduate students may be doing this job to get out of teaching or lab work as part of their own arrangements with their institutions.

Once the editor has picked from their stable of possible reviewers, they basically query whether the reviewers are available and willing.

Mind you, the reviewers are themselves usually professors who - and this is important - receive almost no benefit whatsoever for conducting the peer reviews. No pay. No stipend. Nothing.

(OK, OK, I can hear the objection: reviewing is part of their job as a professor, so they do get paid. To this, I say hogwash. The incentive structure for professors put "reviewing for an academic journal or publisher" at the absolute bottom of the accomplishment scale. Nobody gets a merit raise for number of reviewed manuscripts.)

So, the reviewers are there toiling away for free. Sometimes they do good work. Sometimes they phone it in. To do good work on some scientific papers requires a great deal of care and competence. To do good work in other fields requires a great deal of built up knowledge and discernment.

Then the paper is sent back to the editor with the verdict: publish, reject, or revise and resubmit. The editor passes that on to the submitting author(s), and the cycle starts again.

I want to emphasize that this process is insane. If peer review is broken, it's not merely because these ethically deficient numbskulls are flooding the journals with fake papers, but because the whole thing is built for a nineteenth century academic structure. It's a dumb system even if it were working correctly.

They’re definitely preferable to some random blog or Facebook page. But the problems your raise are very legitimate.

Peer-review is the worst possible mechanism for quality control in the sciences, except for the alternatives that have been tried from time to time.

I agree it's flawed. But in spite of its flaws, it generally works.

The criticism I've gotten from non-academic friends is that it sounds a lot like censorship - a means for established scientists to keep new-fangled ideas that challenge their work out of circulation. But having been on the giving and receiving end of the peer review system for as long as I have, I've seen that it's a lot harder to keep bad science out than to prevent good science from getting in.

The sheer number of journals these days is part of the problem. I've recommended rejection for manuscripts, only to see them published in some other journal a few months later.

From what I've read, the problem with fakery is not science-wide; it's focused on the biomedical and engineering fields. Not saying it doesn't happen across the board, but rarely comes up in my own fields (herpetology, paleontology, evolutionary biology, phylogenetics - I can name some cases, but they're few enough in number that I can pretty much remember them all, and they were all caught by other scientists).

Also - I respectfully disagree with your statement that we professors who review manuscripts receive no benefit. In fact, we do. Firstly, at my institution at least, we're required to keep track of how many reviews we do in a year, and of the journals for which we review. This goes on our CV, and I can tell you from direct experience, if someone isn't reviewing manuscripts, it gets noticed when it comes time for promotion. (This is especially true when one is being promoted from associate to full professor - there's a general understanding that junior faculty won't necessarily have the widespread recognition to be asked. But if someone going up for tenure is reviewing a bunch of manuscripts every year, and if some of those are in Nature or Science, that also gets noticed.)

Secondly - and I think this is more important - we all learn from the process. I obviously can't broadcast what I've learned from a manuscript in review, but knowing about it before it comes out is useful. It helps me avoid starting projects that have already been done, for example. It might make me re-think my own results. Moreover, I take particular delight in reviewing the work of a student; it's another opportunity to teach, which is something I take seriously. And it feels good to help the next generation.

Ultimately, I would ask this - if we're to scrap peer review, what should replace it?

I think there should be a credentialed peer reviewing profession. It would be alt-ac and pay well, and turn academic journal publishing into something more along the lines of ordinary publishing. PhD for peer review and/or industry should be a viable track in graduate programs. Maybe far-fetched, but probably a way out of the current nonsense. It would take a journal of note to pilot such a thing. It would also have the benefit of eliminating most of the ego function of peer reviewing, and the fact that in some fields peer review if blind in name only.

I don't doubt that reviewing is noticed by colleagues and promotion committees, but it is still pretty low on the promotion and tenure totem poll relative to the other things you could be spending your time on (including associate to full). That's part of the problem, and one of the reasons a good deal of peer review is phoned in.

Not in my fields, at any rate.

And there is a credentialed peer-reviewing profession. You can find it at every university, research center, museum, or any other facility that hires professional scientists. I take the system very seriously, as does every professional scientist I've ever encountered.

Peer review is a flawed process, but the issue raised by this article actually has nothing to do with peer review at all. These predatory journals don't really use peer review - if you pay the requested fees, the journal will publish whatever shit they're given. Pay to play. The vast majority of these fraudulent "studies" would never survive peer review. The problem is thus not peer review, but the existence of "journals" that mimic peer-reviewed publications.

And I stand by what I said - in spite of its flaws, there is no alternative.

Maybe you mentioned it, maybe I scanned your post and am a poor reviewer!

Thanks for this, SouthBayDem.

❤️

Apparently some reviewers have been using AI to help them review scientific papers for publication. And some authors have been inserting text instructing AI to provide only positive responses. They accomplish this with plain text instructing the paper submitted for review… in white font inserted in a blank space in the submitted manuscript.

This is apparently a growing problem:
https://www.smithsonianmag.com/smart-news/some-researchers-are-hiding-secret-messages-in-their-papers-but-theyre-not-meant-for-humans-180986996/

compounded by AI.

and scientific rigor. If that system fails to cope with the flood of fake research, it will have dire consequences for scientific research and development in this country. Part of the problem is the "publish or perish" policy of universities and colleges. Perhaps teaching should be prioritized over research for professors who teach.