Skip to Main Content

Exploring the Peer Review Process (Workshop, Summer 2023)

This is the companion guide for the Summer 2023 mini-course workshop.This workshop is an open and critical discussion of the strengths and weaknesses of the modern peer review process, with practical consideration of how it shapes our research & writing

Discussion: A History of Gatekeeping

Questions to consider:

  • Length of peer review process
  • Who are the editor, editorial board, reviewers, audience? Who would they consider their "peers"?
  • Does the journal publisher affect who is accepted?
  • How does journal ranking impact the peer review system?

How Transparent is this Process?

Nature is one of the most highly respected academic journals, and not just in the Sciences.

How do they pick their peer reviewers?

How Anonymous is it?

Academe is highly specialized, and there are only so many experts in a field.
If journals are selecting top researchers in a field, how is it possibly they can't figure out the author of work they're reviewing?
How can the author not figure out who their reviewers were?

The Peer Review Process is Built Upon People

Every actor in the peer review process shapes the process. Not always for the best:

The Publisher may set strict requirements (Ex. # articles per year must be published) that directly shape editorial practices.
Example: The editor of The Journal of Political Philosophy was fired by the publisher in April 2023 for refusing accept & print more articles every year, from 24 to 34 (~40% increase).

The Board of Directors shape journals backed on their personal views. 
Example: In 2020 the Board of Directors CEO of MDPI, which oversees over 100 science journals, used his @mdpi.com email  to share his view that making the scientific community diverse by including "undeserving" scholars hurt scholarship. He also used his position write that "Biden will give more money to support scientists to do research. He will take money from hard working people or tax payers to pay lazy people and there will be less money to distribute as research grants."  

The Editor can override peer reviewers and the board.
Example: In 2017, an article called "the case for colonialism" was submitted to Third World Quarterly, arguing colonialism was good for countries seized into European empires.  The peer reviewers suggested rejecting the article, as did members of the editorial board. The editor published it anyway. Fifteen board members resigned in protest and the article was eventually retracted. 

The Reviewer brings their own views to the process.
Example: In academe there is a trope of "Reviewer 2" who is needlessly mean and picky.
meme showing a "reviewer 2"s extreme requirements for publication

The Author obviously shaped the scholarship they produce, but their knowledge of the peer review process will also effect how they right and publish there work. See Final section.
 

What happens when peer review doesn't work: article edition

Peer review is useful but it's not perfect and flawed research can slip through. When peer review doesn't work for articles, they can be pulled from the journal either by the author or by the journal editors or board; this is called retraction. Articles can be retracted because of honest mistakes but sometimes they are retracted because they are later found to be plagiarized or fraudulent. For example, the thoroughly debunked article linking vaccines with autism was retracted from The Lancet.

The following links have more information on retraction, and the blog and database Retraction Watch keeps track of retracted articles (and if you're using the citation management program Zotero, it uses Retraction Watch's database to flag any retracted citations you may import).