Can we (please) have science without the scientific journals?

We had science before the journals. We can have science after their demise.

Science could be organized as efficiently as restaurants.

When enterprising individuals plan to open a restaurant, do they submit their food to a Culinary Editor who sends it for peer-review by gustatory experts, before customers are allowed to enter the place and try the food?

No, restaurateurs just rent premises, recruit staff, open doors, serve the meals they want at the price they choose, and hope for the best.

Then the review process starts. A whole army of self-appointed reviewers make it their job to describe and evaluate what is on offer. They include specialized magazines, special sections of newspapers or websites. More importantly, a vast number of unsolicited, unpaid Internet reviewers create what an establishment needs – a reputation.

We could – I think we should – apply this model to scientific publishing. Publish first, and then let consumers decide what is good or important and what is not.

This would not mean the end of quality control or peer-reviewing – you can have all of that, indeed you can get more of it and in a more efficient way, if that happens after publication.

And this would allow us to get rid of a rent-seeking intermediary no-one needs anymore: the scientific journals themselves.

What is wrong with the present situation

This section will be short – not that there isn’t much to say, but just that there isn’t much to say that people have not already said. Extensively. Ad nauseam. To wit:

1. As there are more and more journals, the world needs more and more pre-publication peer-reviewing.

2. Since time is unfortunately zero-sum, the opportunity costs of doing peer-reviews are great for scientists, all the more so as this activity is unpaid and unrecognized. (After attending dozens of faculty evaluation meetings, I can report I have never heard the sentence “she does a lot of peer-reviews, that is a very positive point in her favour…”)

3. As that work is imposed on reluctant scholars who often would much rather be doing something else, it is not done very well. (Insert here your favorite horror stories…)

4. Most important: Money is spent where it should not be. Libraries pay exorbitant fees to the remaining old-model scientific journals. Scholars (or their grants, or their institutions) pay large fees to the new open-access journals. We are dealing with a classic case of economic inefficiency brought about by rent-seeking.

Why do you need scientific journals?

Scientific journals combine or combined three distinct functions.

The first function, from the origin, was to turn handwritten letters into publicly accessible information in the form of printed volumes.

The second function, which gradually appeared in the 20th century, was to provide  some quality control, in the form of editor choice, which then became peer-review as we know it.

Third, as a side-effect of peer-review, reputation differences between journals started to provide evaluation, that is, some information about the relative quality of what was published. If your description of Norwich beetles was of negligible importance, you were more likely to see it published in the Norfolk Archives of Entomology than in Cell.


But then a miracle happened.

In the modern world (which started around 1990), scholars and scientists now have the means to present their results in the shape of an elegantly formatted PDF file that they can post on a webpage, thereby making it available to the entire planet at a vanishingly small cost.

So most individuals can perfectly well serve the first purpose of journals from the comfort of their homes or offices. But what about the second and third roles of journals, scientific quality control and evaluation?

 

A modest proposal: Ex post facto peer-evaluation

There are many ways in which we could get quality control and evaluation a posteriori, that is, after the appearance of a research report in cyberspace.

Consider these possible situations:

Individual comments. People would simply send each other pointers to important, exciting results (or dull, incoherent proposals) as an expression of their own judgement. This already happens in the form of, e.g., Twitter threads about various publications or conference talks. As usual with such situations, the commenters’ reputation modulate the effect other recommendations, but also vice-versa, as endorsements can affect people’s reputation.

Spontaneous communities. Individuals could pool resources and constitute a group with a common identity, the point of which is to comment on current productions in their chosen field. To take a random example: Individuals who work in cognition and culture are worried lest many important contributions to the field are not recognized. They get together and publish a monthly review of notable publications in the field, with appropriate scholarly comments. They can also, if they so wish, maintain a Wall of Shame to denounce gross distortions or terrible scholarship.

Existing institutions. It would become one of the tasks of official scientific institutions, from the humblest to the most exalted, to offer their services to science in the form of a posteriori evaluation of scientific reports. For instance, the Department of Materials Sciences at Camford University is supposed to promote and communicate, to students and the public, the best of Materials Science. One can imagine that such a department could invest modest resources and enhance its public profile, by maintaining a list of officially approved papers in Materials Science.

There could be many other possible models.

Note that, in these different models, you can get all the traditional peer-reviewing and evaluation that you want, and in fact get more of it, since you would get it at a lower price. If you are worried that people report results with p-values and no effect size – you can point it out. If you think that a particular study’s results actually undermine the conclusion – let people know. If you judge a finding to be more consequential than better-known studies, nothing will stop you from saying it. If you want to make fastidious comments about the fact that the tables should be turned into charts and the charts should be tables – be my guest.

And, in constrast to the present situation, the low-quality comments and pernickety obfuscation that mar so many peer-reviews would meet the appropriate sanction, in the form of damaged reputation for their authors. To avoid this, people would try to sound insightful and creative, which would make most of us more creative and insightful.

For less money.

Pessimistic note: Rent-seeking is a tenacious weed

As I said, scientific journals these days benefit from rent-seeking.

When such situations occur and persist, it often is because of coordination problems.

In this case: Who would want to risk their reputation and career by publishing their articles on their own website, waiting for the scientific community to pass judgment? In a reputation-based community (like all science communities), publication in high-status journals is seen as a reliable, easily-detected signal of (difficult to detect) underlying quality. And, as we know from animal communication, deciding not to send a signal about one’s own qualities is equivalent to signaling low quality.

Which is why, in all likelihood, we may be stuck with scientific journals for some time yet.

So our only hope is that communities and organizations jump start the process. We could for instance start a Cognition and Culture Review, that points people to various papers and studies, regardless of where they appeared or, indeed, whether they were “published” at all in the traditional sense.

Post-Scriptum

For readers interested in food: Actually, there used to be places where restaurants were peer-reviewed, in a manner of speaking, and their menus approved by competent specialists before being offered to the public. That was in socialist countries. And we know the result. No-one waxes nostalgic about the great restaurant scene in 1960s Bucharest. QED.

2 Comments

  • Andrew Buskell
    Andrew Buskell 28 June 2018 (12:02)

    Network Effects and DiscriminationI have a general worry with an outright abandonment of scientific journals.

    A serious problem with the models discussed above concerns the way in which it would support network-based effects and problems. Consider that one of the main concerns about current academic life is systematic discrimination (against women, people of colour, from non-Western universities, etc.). Some of this discrimination is plausibly contributed by network effects, notably, where researchers promote work of researchers they know or whom have esteem, while ignoring or denigrating work of other researchers.

    While it is true that many prestigious journals have a long history of their editorial boards being run by white, western males—this is something that is changing and being compensated for (more on this below). My worry is that in abandoning scientific journals one would exacerbate network effects insofar as positions of power in academia continue to be predominantly held by such individuals who can perpetuate members in their chosen networks.

    This leads me to a second point. Something missed among the reasons for adopting the scientific journal model (and journals more broadly) is accountability. When there is gross misconduct, data manipulation, or poor publishing practices, journals (more specifically, editorial boards) can be held responsible. So, for instance, they can be held responsible for systematic discrimination and react accordingly—perhaps by instituting more robust blinding procedures. This can lead (little by little) to greater representativeness in high-profile journals by individuals who have traditionally been discriminated against.

    On the laissez-faire models suggested here, there is little recourse for responsibility, or mechanisms for addressing such systematic discrimination.

  • Alberto Acerbi
    Alberto Acerbi 4 July 2018 (10:33)

    Avoid submitting to for-profit journals seems a lost battle. What about avoid reviewing?
    Thank you for this post! I fully share the frustration with the traditional academic publishing system, and I agree that coordination problems are the main culprit in the current situation. Nobody – almost nobody – can afford not submitting to for-profit publishers. If I’d feel any of the drafts I am working at the moment is good enough, I will myself run to submit it to Nature Human Behaviour or similar…

    Said so, there is perhaps another possibility. I recently discovered that Behavioral Ecology (the journal) gives reviewers a lifetime 25% discount on OUP books. My first reaction was to think to stop reviewing for for-profit publishers that do not compensate reviewers, but the next step would be, in fact, to stop reviewing for for-profit publishers at all.

    This would not be a cost for us (if anything, it would save some time), and, if enough people would avoid reviewing, it would start to be also inconvenient to submitting to those journals as manuscripts would get stuck for a long(er) time. Also, scholars that would keep reviewing will possibly receive more and more articles, so that they may change their mind.

    Notice I am not saying that peer-review is wrong or useless (I do not think so, and it is anyway irrelevant for my point) or that reviewers must be paid (I do not think so: in a fair system, reviewing should be one of the various activities of researchers, and we are clearly not paid singularly for each of them), but that this may be a way to break the vicious circle that makes scientists complain with one hand and submit with the other.