Reviewing

I have been doing a crapload of reviewing lately. This is okay, since I took a little hiatus from reviewing during my busiest tenure push, so I feel good about giving extra back now. But I've started to notice some weird things about reviewing in such a small field.

One, if the manuscript is lousy then it's not a given that I know who the author is. Often the author might be a new graduate or even a grad student or something like that. But if the manuscript is very good, these days I can almost always tell who the author is, even though it's supposed to be blinded. Our field is just too small. I tend to get asked to review in areas about which I know a lot, so I've published in those areas and I've read the literature. If somebody is writing in one of those areas, I can recognize the work. I know it'll be one of two or three people who has done work like that. As soon as the manuscript begins to include blinded cites (Author, 2010; Author, 2008) then I say, oh, I know that so-and-so published that 2008 paper about that topic.

Anyway, I guess what I'm saying is that our field is really too small for genuinely blind reviews, half the time. I reveal my likely knowledge of the author less and less these days. I resolve that by not trying to specifically pin down the author. I might have a strong suspicion about who it is, but in most cases I can reasonably say that I'm not 100% sure. If I don't have any close relationship with the likely author, I can be objective. Occasionally I'll run across something in which I know without any doubt who the author is, definitively. (For instance, if somebody doesn't blind something well enough, and refers to a specific study about a specific student's reasoning, and I'll recognize the student's pseudonym). In those cases, I reveal my knowledge to the editor and let them to decide whether I should continue the review.

I'm reviewing a very good manuscript right now. You can tell it's great when you find yourself wishing you had written it!

I recently reviewed a very poor manuscript. I got extra pissed off when the author(s) discussed a recently-published study of mine in a way that revealed that not only had they not read the paper, they hadn't even read the abstract. They mis-characterized the paper so grossly that reading 2 sentences into the abstract would have revealed their error. That's just incredibly sloppy. You shouldn't discuss papers in your lit review that you have obviously never read.

I've begun to notice that often I can tell pretty quickly into a read whether the manuscript should be accepted or rejected. If the intro and lit review is crappy, then there's little chance the study will be well-constructed enough to merit acceptance. In contrast, a very well-written intro tends to reveal a carefulness of thought that is often associated with a well-designed study. However, occasionally I end up changing my mind by the end of the manuscript, so it's best to keep an open mind throughout.