Jimmy Higgins

Had a question regarding peer review journals.

How exactly is it done? I'm sure there are probably a few different ways:

1) Does the reviewer know who the author is?

2) Does the journal know who they are going to have review it? (is it a cycle, random, or do they just put it in the mail and hope for the best )

3) Does the author know who reviewed it?

4) What keeps the journal from "cheating" the system by sending certain papers to "favorable" reviewers?

Just had these questions seeing that TJ seems to be the discussion in TWeb right now. I know TJ is illegitimate, but I'd like to know what makes real journals legitimate. Thanks.

Wounded King

1) It varies depending on the Journal.
2) They will choose people in a relevant field, so they determine who reviews it, no randomness involved.
3) The author usually will not know who the reviewer is, but sometimes they can tell anyway .
4) Nothing, except for their own self interest in keeping the standard of the material in their Journal high.

roxrkool

'Cheating' has been known to happen. Nothing is perfect, but so far it's the best way.

Also, the cheating that has been known to happen tends to center around specific fields. I think the medical field is one such field that is considering revising their peer-review process. That's just something I read somewhere, so I don't know exactly what happened, but I suspect it has something to do with drug companies and financial 'incentives.'

RufusAtticus

My advisor is an associate editor of Genetics, and she has passed much of the clerical duities to Wildy and I in the past.

1) Does the reviewer know who the author is?

From my experience, yes.

2) Does the journal know who they are going to have review it?

Usually, the papers my advisor gets are in her field or close to it and she looks at the paper and determines based on her knowledge what scientists would be appropriate reviewers. She then contacts them and asks if they can review it. It is very common that they can't so she has to keep going down the list she has made up for the paper. Sometimes she looks at the references in the paper itself to find reviewers. Also authors can recommend reviewers if they feel that the topic of their paper is narrow and the editor might not know where to look. Authors also can request that certain people not review their paper. This is usually done because they have competition and they don't want their competition reading the paper before its out.

3) Does the author know who reviewed it?

Reviews are anonymous, unless the reviewer wants to be known.

4) What keeps the journal from "cheating" the system by sending certain papers to "favorable" reviewers?

Time. Most scientists don't have the time to do many reviews in a year. Thus it would be difficult to have one for rubber stamping every paper. Also the critical nature of science discourages editors from allowing crap through.

Jayjay

Of course, AiG uses the potential for "cheating" as a proof that all journals in every field, and in every country of the world are "cheaters" and committed to their materialistic uniformitarian agenda to promote atheism or something.

RufusAtticus

TJ isn't even peer-reviewed. An editorial board reviews all submissions. Guess who's on the editorial board. I suspect that it is populted by AiG members like Ham and Sarfati, thus I doubt any of the people who regularly write for TJ get any sort of review at all.

Undercurrent

I've done some reviewing for the Jornal of Mathematical Physics, so I can give answers from my experience. Your mileage may vary.

Yes. I've never reviewed the paper of someone I've known personally, but always had the name/address/&c.

Every so often, a paper pops up in my mailbox with a note that says "please review this, or if you're too busy, tell us so we can get someone else to do it". The papers I've reviewed have all been related to papers that I have had published in the same journal, so that's probably a selection criterion.

Not to my knowledge. I've never been required to be anything but anonymous.

The fact that no one can really have a strong ideologically-motivated bias about the asymptotic properties of hypergeometric functions.

That last bit is probably more important than my glib rendering indicates. Most papers are about single measurements an analyses. It's hard to have an ideologically-biased single measurement. There are a whole bunch of little papers rather than a few large papers. The fact that everyone is so focused inhibits any large scale conspiracy.

beausoleil

I've done a fair bit of reviewing, so I'll chip in and maybe you can spot the variety of approaches. I work in geochemistry.

Always, in my experience. Almost all the papers I've reviewed have been by people I've at least met. Many by people I know quite well.

Many journals have a board of associate editors who are academics at universities. They send the paper to the AE closest to the field, and he selects a few (2-3 reviewers) and recommends publication or not. Science and Nature, however, seem to use full time Professional editors to choose the reviewers and make the decisions.

Many journals publish a list of reviewers who've contributed over a year - this is vague enough to preserve anonymity.

At the reviewer's discretion. I always used to 'fess up, but had a bad experience with being lobbied incessantly by an author and since then have been more cicrumspect.

They can, but somebody else will rip anything weak to shreds in a competing journal and the situation self-rectifies.

Valentine Pontifex

Well they claim to be peer reviewed.

;-)

Wounded King

Well as long as your peers consists of creationists and ID enthusisasts then there no problem. You only need a couple of dozen completely uncritical peer reviewers to produce a peer reviewed Journal that noone in Science is going to take seriously.