Can we (please) have science without the scientific journals?

We had science before the journals. We can have science after their demise.

Science could be organized as efficiently as restaurants.

When enterprising individuals plan to open a restaurant, do they submit their food to a Culinary Editor who sends it for peer-review by gustatory experts, before customers are allowed to enter the place and try the food?

No, restaurateurs just rent premises, recruit staff, open doors, serve the meals they want at the price they choose, and hope for the best.

Then the review process starts. A whole army of self-appointed reviewers make it their job to describe and evaluate what is on offer. They include specialized magazines, special sections of newspapers or websites. More importantly, a vast number of unsolicited, unpaid Internet reviewers create what an establishment needs – a reputation.

We could – I think we should – apply this model to scientific publishing. Publish first, and then let consumers decide what is good or important and what is not.

This would not mean the end of quality control or peer-reviewing – you can have all of that, indeed you can get more of it and in a more efficient way, if that happens after publication.

And this would allow us to get rid of a rent-seeking intermediary no-one needs anymore: the scientific journals themselves.

What is wrong with the present situation

This section will be short – not that there isn’t much to say, but just that there isn’t much to say that people have not already said. Extensively. Ad nauseam. To wit:

1. As there are more and more journals, the world needs more and more pre-publication peer-reviewing.

2. Since time is unfortunately zero-sum, the opportunity costs of doing peer-reviews are great for scientists, all the more so as this activity is unpaid and unrecognized. (After attending dozens of faculty evaluation meetings, I can report I have never heard the sentence “she does a lot of peer-reviews, that is a very positive point in her favour…”)

3. As that work is imposed on reluctant scholars who often would much rather be doing something else, it is not done very well. (Insert here your favorite horror stories…)

4. Most important: Money is spent where it should not be. Libraries pay exorbitant fees to the remaining old-model scientific journals. Scholars (or their grants, or their institutions) pay large fees to the new open-access journals. We are dealing with a classic case of economic inefficiency brought about by rent-seeking.

Why do you need scientific journals?

Scientific journals combine or combined three distinct functions.

The first function, from the origin, was to turn handwritten letters into publicly accessible information in the form of printed volumes.

The second function, which gradually appeared in the 20^th century, was to provide  some quality control, in the form of editor choice, which then became peer-review as we know it.

Third, as a side-effect of peer-review, reputation differences between journals started to provide evaluation, that is, some information about the relative quality of what was published. If your description of Norwich beetles was of negligible importance, you were more likely to see it published in the Norfolk Archives of Entomology than in Cell.

But then a miracle happened.

In the modern world (which started around 1990), scholars and scientists now have the means to present their results in the shape of an elegantly formatted PDF file that they can post on a webpage, thereby making it available to the entire planet at a vanishingly small cost.

So most individuals can perfectly well serve the first purpose of journals from the comfort of their homes or offices. But what about the second and third roles of journals, scientific quality control and evaluation?

A modest proposal: Ex post facto peer-evaluation

There are many ways in which we could get quality control and evaluation a posteriori, that is, after the appearance of a research report in cyberspace.

Consider these possible situations:

Individual comments. People would simply send each other pointers to important, exciting results (or dull, incoherent proposals) as an expression of their own judgement. This already happens in the form of, e.g., Twitter threads about various publications or conference talks. As usual with such situations, the commenters’ reputation modulate the effect other recommendations, but also vice-versa, as endorsements can affect people’s reputation.

Spontaneous communities. Individuals could pool resources and constitute a group with a common identity, the point of which is to comment on current productions in their chosen field. To take a random example: Individuals who work in cognition and culture are worried lest many important contributions to the field are not recognized. They get together and publish a monthly review of notable publications in the field, with appropriate scholarly comments. They can also, if they so wish, maintain a Wall of Shame to denounce gross distortions or terrible scholarship.

Existing institutions. It would become one of the tasks of official scientific institutions, from the humblest to the most exalted, to offer their services to science in the form of a posteriori evaluation of scientific reports. For instance, the Department of Materials Sciences at Camford University is supposed to promote and communicate, to students and the public, the best of Materials Science. One can imagine that such a department could invest modest resources and enhance its public profile, by maintaining a list of officially approved papers in Materials Science.

There could be many other possible models.

Note that, in these different models, you can get all the traditional peer-reviewing and evaluation that you want, and in fact get more of it, since you would get it at a lower price. If you are worried that people report results with p-values and no effect size – you can point it out. If you think that a particular study’s results actually undermine the conclusion – let people know. If you judge a finding to be more consequential than better-known studies, nothing will stop you from saying it. If you want to make fastidious comments about the fact that the tables should be turned into charts and the charts should be tables – be my guest.

And, in constrast to the present situation, the low-quality comments and pernickety obfuscation that mar so many peer-reviews would meet the appropriate sanction, in the form of damaged reputation for their authors. To avoid this, people would try to sound insightful and creative, which would make most of us more creative and insightful.

For less money.

Pessimistic note: Rent-seeking is a tenacious weed

As I said, scientific journals these days benefit from rent-seeking.

When such situations occur and persist, it often is because of coordination problems.

In this case: Who would want to risk their reputation and career by publishing their articles on their own website, waiting for the scientific community to pass judgment? In a reputation-based community (like all science communities), publication in high-status journals is seen as a reliable, easily-detected signal of (difficult to detect) underlying quality. And, as we know from animal communication, deciding not to send a signal about one’s own qualities is equivalent to signaling low quality.

Which is why, in all likelihood, we may be stuck with scientific journals for some time yet.

So our only hope is that communities and organizations jump start the process. We could for instance start a Cognition and Culture Review, that points people to various papers and studies, regardless of where they appeared or, indeed, whether they were “published” at all in the traditional sense.

Post-Scriptum

For readers interested in food: Actually, there used to be places where restaurants were peer-reviewed, in a manner of speaking, and their menus approved by competent specialists before being offered to the public. That was in socialist countries. And we know the result. No-one waxes nostalgic about the great restaurant scene in 1960s Bucharest. QED.