Favouritism and Peer Review

Brian Leiter links to a fascinating article on 'The 'Black Box' of Peer Review', which includes the following mind-bender:

[W]hen it comes to an affinity for work that is similar to their own or that reflects personal interests having nothing to do with scholarship, many applicants benefit in a significant way. In a passage that may be one of the most damning of the book, Lamont writes: "[A]n anthropologist explains her support for a proposal on songbirds by noting that she had just come back from Tucson, where she had been charmed by songbirds. An English scholar supports a proposal on the body, tying her interest to the fact that she was an elite tennis player in high school. A historian doing cross-cultural, comparative work explicitly states that he favors proposals with a similar emphasis. ... Yet another panelist ties her opposition to a proposal on Viagra to the fact that she is a lesbian....

Seriously? There are academics who consider such autobiographical facts to provide a (fundamental) reason for favouring one grant proposal over another? That would be insane. But perhaps that's too uncharitable an interpretation of what's going on here.

Perhaps they are instead taking their personal experiences as providing them with special epistemic access to an independently existing reason. The lesbian, for example, believes that medical research is excessively "focus[ed] on men" -- which, if true, would provide a perfectly objective reason for shifting funding elsewhere. The fact that she's a lesbian might make her more sensitive to this reason (or possibly even oversensitive, if her perceptions turn out to be inaccurate); but that's not at all the same thing as claiming that her being a lesbian is itself the reason to oppose the grant.

Similarly in the other cases: it would be insane to take the mere autobiographical fact that you personally were recently charmed by songbirds as a reason for supporting research on songbirds. But perhaps one could reasonably hold that one's experience put one in a position to recognize a more objective fact: that songbirds are charming, and thus worthy of further study. The autobiography gives a causal explanation of how it is that one came to be aware of this reason. It is not itself the "reason" or consideration that counts in favour of the grant. (One might say, "If it weren't for my recent trip to Tuscon, I would never have appreciated how objectively worthy this grant proposal is!")

I'm especially suspicious of the blanket assumption that it's some kind of disreputable "favouritism" for academic evaluators to prefer work that is similar to their own (in topic or methodology). It's possible for such preferences to be disreputable, and perhaps even likely that they involve some degree of motivated reasoning or bias. But let's not forget that academics' own work is shaped by their prior evaluations. I work on X in part because I think it's one of the most pressing and interesting issues around. I use methodology Y in large part because I think it is the most rigorous, fruitful, or reliable way to solve the problem. Other methodologies (consulting a magic 8-ball, say) I avoid because of my prior judgment that they're not good methods of inquiry. Is it thereby "favouritism" to prefer a grant proposal that uses methodology Y rather than the method of magic 8-ball consultation? Surely not.

To be "favouritism" of the odious kind, it must be that one's judgments are tracking one's personal interests independently of their merit. Here's a test: any fair-minded academic can presumably identify additional topics or methodologies that they judge to be roughly on a par with their own in terms of objective merit/interest. We can't do it all, so what we end up specializing in is presumably just a small subset of the areas we take to legitimately merit such interest. A fair-minded evaluator will thus be just as receptive to proposals in those other meritorious areas as they are to proposals in their own meritorious area. But it is no part of fair-mindedness that one must also be receptive to work one considers intellectually bankrupt. (At least if that opinion is itself reasonable. One should, of course, be wary of adopting such a dismissive view on insufficient evidence!)