Publish Then Filter

A common theme in analyses of the Internet is the transformation from a 'filter then publish' to a 'publish, then filter' world. The high costs of publishing previously forced the former model on us: anyone who wanted to see their work in print first had to win over the gatekeepers (editors of newspapers, journals, etc.). But now anyone and his dog can publish for free over the Internet. So the contemporary challenge is post-publication filtering, i.e. how to find the gems in the torrent of information out there.

One option is to turn to the old gatekeepers for guidance. Anyone can self-publish, but not everyone can publish in the pages of the NY Times or Nous. So we can keep ourselves in a world of informational scarcity if we limit our attention to particular locations which impose pre-publication filtering.

That's fine as far as it goes, but it is an extremely conservative response to the new information ecology. It makes us no worse off than before, at least. But it's worth raising the question: might we have an opportunity now to improve the way we do things? I've already mentioned open-access, which is of course a no-brainer. But that is just a minor tweak, still firmly within the old 'filter then publish' paradigm. To be clear: I think there is an important place for this, at least for the foreseeable future. But I wonder whether we could supplement this with some form of more widely distributed post-publication peer review.

I imagine, for example, the Philosophy Papers Online database could be expanded to allow registered philosophers to rate and/or review the papers found therein. (If measures are needed to 'guard the guardians', these reviews themselves could be subject to peer review -- Slashdot style -- and weighted accordingly. Other online communities have already solved the technical question of how to create a software infrastructure that supports peer production. All we have to do is implement it.)

This would make PhOnline a vastly more valuable resource, since users could browse the most highly rated papers, using these peer ratings as a guide to the most important new scholarship. (At present, users may search by author, title, or date, but there is no way to gauge quality.) This would only work if other philosophers put in the effort to review their colleagues' work. But we already do this for journals, so I don't see why we wouldn't also do this for each other. Depending on how it's set up (i.e. not anonymous review), there could be additional incentives to perform this service, as quality reviewers would benefit from reputational gains within the profession. It could even be technologically enforced, e.g. by requiring that users offer a few reviews before they are allowed to submit another paper of their own to the database.

(In that case, perhaps it would be best to start this project from scratch, rather than trying to build upon an already existing database of papers.)

John Holbo offers a variation on this sentiment (but restricted, I take it, to work that has already passed through the old channels of official credentialing -- I would want to expand this to "unpublished" drafts):

If overproduction is inevitable, which I grant, the primary question is not how to fund it but how to ameliorate the damage it does us. (Having gone overboard by describing excess scholarship as 'effluent' I should probably add: producing things no one wants to read is perfectly harmless so long as these undesired things do not collectively block the road.) The question is how to overproduce with intellectual dignity?

The answer, I think, is that a supplement is needed to a pre-publication peer review process that inevitably hyper-produces hypertrophic 'conformist excellence within the heuristic contraints ...' The supplement should be a hyper-efficient post-publication peer review process that tells you what you might actually want to read.

A simple normative principle. Every scholarly book published in the humanities should be widely read, discussed and reviewed - should have it's own lively blog comment box, not to put too fine a point on it. Because any scholarly book incapable of rousing a modest measure of sustained, considerate, intelligent chat from a few dozen souls who specialize in that area shouldn't have been published as a book - i.e. after several years labor and an average production cost of $25,000. Turning the point around: any book worth that time and expense, that fails to be widely read, discussed and reviewed - that is not given it's own blog comment box - has been dramatically failed by the academic culture in which it was so unfortunate as to be born. [...]

Why is this really quite low normative standard of healthy discussion not presently met? The technological barriers are non-existent, the financial barriers negligible. It's cultural dysfunction. Sheer institutional sclerosis.

The Real Circulation Problem - of which low book sales are a symptom - concerns ideas, not paper. The academic humanities have simply never grown hyper-efficient networks for post-publication peer review that are remotely adequate to the excessive volume of peer-reviewed scholarship generated, especially in just the last few decades. This is the real scholarly argument for moving aggressively online, although it is bolstered by many economic arguments. As I have written before, the beast has poor circulation. The only way to get the blood of ideas moving is to rub its sorry limbs vigorously with ... conversations. Intelligent, bloggy bookchat by scholars, to label this crucial ingredient as the essentially unpretentious thing it is. That isn't scholarship; but - in a world with too much scholarship - it may be an indispensable complement to scholarship.

Cf. Tyler Cowen, for a more radical long-term projection:

I don't envision the free access system as the status quo but free. Papers would be ranked directly in terms of status and popularity rather than ranked through the journals they are published in. Ultimately there wouldn't be journals and this would make a big difference as journals are the current carrier of selective incentives and status rewards...

I'm not sure about this -- what about blind review? We'll presumably want to retain this somehow, if not by journals than via some similar formal means of competition. (Though others have suggested that googling spells the doom of blind review, in which case it's hard to imagine why journals would survive in the long term.) In any case, journals certainly aren't going anywhere in the short term. So I'm really proposing a supplementary system (not a replacement) that I think we'd all benefit from right away.

Update: I've shifted discussion of my specific proposal to a new post.