Some of the publishers out there are just outrageous. (This is the pitch: “You do all the work, for which we charge you £2,700 per article. And don’t worry, this money is not wasted, it will end up as a 40% profit margin for us.”)
13 Likes
Yea, a captive market can do that. The pushback will be slow. Getting tenure in most fields comes down to publishing multiple times in a select group of journals. Even if the costs increase 100x, it is still financially “worth it” if not doing so leads to unemployment. Sort of surprising that this didn’t happen sooner, honestly. I guess previously the publishers cared less about profits.
EDIT to add: Individual academics can’t be the people to fight back either. It needs to be done by the people paying the big bucks (universities, mostly). Some have started.
7 Likes
Interesting take.
I have seen the reasons for what you say about publishing in the right journals: it is the personal reviews, where the reviewers just can’t be bothered to assess the quality of the papers themselves. They just count the number of the papers that appeared in the “good journals”.
But this can change. This sort of publication fees is something that is getting harder and harder to justify to funding agencies.
5 Likes
Great mention of exorbitant e-book prices, went through this as a student though it hadn’t gotten as bad as >10x markup over physical copies. For comparison, contemporary video game files are much bigger than any e-book’s, and the industry standard price going from 60 to 70 dollars is widely regarded as opportunistic price-gouging during widespread inflation. Yet by e-textbook standards, this is a bargain.
I understand the need for profits, but institutions need to do more to reduce the burden on students. But they can actually be incentivized to do the opposite. One especially dodgy practice I’ve seen is the professor publishing their own textbook with a new edition every year. There were virtually no changes to the text, except large permutations of the homework problems. To spell it out, he was essentially selling his own homework at inflated prices to his yearly students instead of selling the otherwise unchanged textbook at a lower price to a wider customer base. On one occasion he slipped and bemoaned the low sales caused by piracy (students secretly sharing one new edition for the homework). I’m sure the newer anti-piracy mechanisms made him happier.
3 Likes
I’m in general getting more and more skeptical of how governments grant funding. There are, for example, some heavy criticisms that can be made of peer-review such as that science is a strong-link problem (Science is a strong-link problem - by Adam Mastroianni) meaning that science improves due to the availability of great papers and not due to the lack of poor papers, so hindrances like peer-review only hinder this. Also, I’ve heard of cases where peer-reviewers see themselves as competitors and drag the review process along for years. And why wouldn’t they from a purely competitive standpoint? Furthermore, publications in arXiv have shown to provide value irregardless of peer-review.
7 Likes
I think this is a partially good argument. However, peer review value is also in that it makes the reviewed articles better (when it works, that is understood).
Also, not letting through poor science is very important because science gets used (abused?). Take for instance the MMR vaccine scandal: if properly peer-reviewed, the article(s) by Wakefield might have been filtered out. As it were, those articles remained in the pool of the “medical knowledge”, which led to substantial harm.
6 Likes
I have to disagree, it’s a very tall order to expect a few distant readers to catch academic fraud from one paper. The issue with the MMR paper isn’t its low sample size and unsubstantiated hypotheses; pilot studies routinely provide insight into methods and intended future research. It’s the same reason doctors and academics gave Wakefield chances to do further research to substantiate his claims (which he did but he suppressed the negative data and harms to children). The issue was the coordinated fraud, malpractice, media overhype, and public ignorance of science. If the Lancet could have done anything, it would perhaps be retracting things as soon as the lead author decides to evade replication studies by doing media tours.
2 Likes
There are certainly those who think the original study should not have been published: Lancet retracts 12-year-old article linking autism to MMR vaccines - ProQuest That could have been accomplished with a thorough peer review.
1 Like
That paper was peer-reviewed, and it was impossible for the reviewers to know about the lawyer paying Wakefield for results, his rejected patent for an alternative measles vaccine, the falsification of medical data, or the other authors’ ignorance of the medical claims (it would take 6 years for them to publicly denounce the study, only after journalist Brian Deer unearthed conflicts of interest). As far as they knew, this was a routinely small sample that the authors all believed to indicate a medical phenomenon that was worth investigating. You could add 10x more reviewers and given them 10x the time, and all they would see is a possibly overconfident pilot study. Peer review only stems implausible research, not wrong research or fraud; those are unknowingly peer-reviewed and published all the time. Note how the people in that article criticizing the Lancet for publishing Wakefield’s study did not suggest anything regarding peer review, even with the benefit of hindsight. Overconfidence in peer review was in fact a major factor in COVID misinformation spreading among the public.
A reliable result comes from replication in larger sample sizes and across independent labs. In fact, multiple publications is how fraud gets caught sometimes. For example, frauds can slip up and duplicate generated data and graphs across several papers.
2 Likes
I rather thought Dr. Lewis was quite clearly pointing to problems with peer review (“… why Lancet published …”).
In any case, my point that peer review has roles to play beyond those considered in the “link” blog does not hinge on the specifics of a single case.
Publication is not synonymous with peer review. Consider that there are many journals that publish without any peer review, or that submissions can be rejected before any peer review or despite positive peer review.
Calls for improvements to peer review always come after extensive public health consequences or massive wastes of research funding, but this is just reactive finger-pointing. It is invariably wishful thinking, not any actionable suggestions to stem the tide of wrong or fraudulent claims, even those that don’t get traction. Peer reviewed journals have never promised that they only publish correct information, and that’s just something people need to accept. There’s a reason that the replication crisis is about methodology, while the peer review crisis is about labor.
1 Like
The quoted price is for university libraries, though, not for individuals. The publishers want to recoup the cost of many students checking out the book over time. (To be clear, I don’t agree with this practice.)
I would like to see a model of how many people have to “defect” from the current system of trying to publish in the most prestigious journals before it collapses. And how likely this is to happen. As it is, people don’t want to play chicken with their own careers, as long as your colleagues don’t stop you can’t either.
What I don’t understand is why public funding agencies don’t actively work more to improve the situation and, for example, update their funding guidelines to deemphasize the journals that applicants published in. After all, public research should benefit the public, so free to read at the least. Maybe it’s because funding agencies are just not good at assessing the quality of someone’s research because that’s just generally a hard thing to do, so the journals are welcomed as a simple heuristic.
4 Likes
There will always be papers slipping through since there is no formal way to always accurately establish whether a paper is “right” and since researchers are a critical bunch it doesn’t matter. Maybe some paper temporarily gets too much attention due to hype, but that should peter out when others disprove it. Just like it does in a world of peer-review.
1 Like
People have always liked profit. The main difference is there’s no longer competition–prior to WWII peer review was almost unheard of, and since then it’s just gotten more and more important to publish in the same handful of prestigious journals.
(The main exception being Machine Learning, where pretty much everything important is just on Arxiv.)
1 Like
I think that’s actually a great example of why peer review is a pretty bad system–it gives bad papers a false veneer of scientific credibility. If Wakefield had posted his paper on BioArxiv, nobody would’ve cared. Instead, it got a big stamp of approval from The Lancet (probably the hardest journal to get into in the entire field of medicine). If even The Lancet isn’t doing a good enough job to stop a “Study” of about a dozen children with autism with no random sampling from being published, I’m going to go out on a limb and say it’s just not fixable.
1 Like
Turns out he had an agenda, and getting published on BioArxiv would have worked just as well for him (and his sponsors), if BioArxiv had the same level of acceptance then as it has now.
This is interesting of course: How does a reader know that the paper on Arxiv is any good? Especially now, when to get a paper written on machine learning, with all the bells and whistles, all it takes is sixty seconds on ChatGPT.
Very possible. The main factor in the hysteria’s success was the uninformed mainstream press’s amplification of his agenda without proper scrutiny. One facet of the narrative was that he was an independent doctor who wasn’t muzzled by methodological rigor, someone who “listened to the parents.” Being rejected by mainstream journals might have actually played into that narrative.
I think the real problem is people overestimating that credibility. Dr. Richard Horton, the Lancet’s editor-in-chief throughout the Wakefield fraud, has actually written an article “What is medicine’s 5-sigma” that makes the bold claim:
The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. As one participant put it, “poor methods get results”.
Peer review is useful for filtering out bad practices, but it does not filter out all of them. Some bad practices may fundamentally be impossible to detect by 3 people from 1 isolated manuscript.
1 Like
Please don’t get me too much started on the topic, such a painful one.
I am getting paid by the state to do research without any supervision from the said state (i. E. independent researcher). I send my research to a journal. Some university professor, who gets salary from the state receives my paper. Sends it to other professors on state salary, they spent time on it. NO ONE in this equation so far is being funded by a publishing house! Finally, after the paper gets published I am giving all of my copyrights away and no one from any country, which payed for the whole process may read it unless they pay some private company?! Such horrible BS.
I don’t understand why this topic steered towards peer review, peer review has nothing to do with this BS. The journals, for the most part, have nothing to do with the publisher. In Economics most publishers do not even pay the reviewers***
And yes, as mentioned, you HAVE to play the game if you want anything to do with that career. You have to publish and publish well, as this is pretty much the only thing everyone care about. Most people don’t care at all about what you actually do/study! I cannot even recount the amount of times I have been to research seminars where the talk giver is NOT introduced as “person X’s research helps us understand the labour market better” but as “person X’s research has been published in these high journals”!
Also, you send a paper to a journal and just by pure chance within two weeks, you get asked to review a paper from that journal and you really feel like you have to accept it.***
It is the large coordinated universities who have the most power. And not just one or two but it must take a coordinated effort. For example Harvard at some point declared war on Els… and their researchers didn’t have to publish in those journals but let’s face it - if I am already a senior researcherser in Harvard I probably don’t have as much to prove…
*** Disclaimer, I have reviewed for journals who are owned by the One from the article multiple times and I have also been renumerated.
10 Likes