Reviewing the effectiveness of peer review process – Scirev

scirevHow efficient is peer review in scientific journals? How long time does it take to get the reviewers’ comments? Were the comments useful? What is the time period you can count with from submitting to publication? How was the review process in general?

We have information about the impact factors of the journals according to various measures and indicators, and one can check up the authors’ impact factor or the influential articles of a journal in Publish or Perish. However, these things give information about the articles that have been published.

At the same time, there is little information about the peer review processes in the various journals and the time it takes. Being a senior researcher, one usually gathers this knowledge over time through experience. However, these are questions that become more and more relevant also for PhD students (and ambitious Master graduates who want to write something from their thesis), as the article-based PhD is gaining prominence in a number of countries. As long as the dissertation is dependent on articles being published, these questions about the review process and the time it takes, can have a significant impact on the candidates writing process and time management.

This far, this kind of information passes between colleagues based on first hand experiences and maybe some good recommendations from others. You might hear that some journals are particularly slow in processing, whereas others are known for their effectiveness, and furthermore, this seems to vary from time to time. As such, the information is rather fragmented and incomprehensive.

To get some more systematic information, the website SciRev might become relevant. The ambition of SciRev is to make the review process more transparent by sharing information about the review process, its quality and the time it has taken – both for articles that have been published and those that have been rejected. By this, they hope to incentivise the journals to be more efficient and to critically examine their editorial and review processes.

Sure, these kinds of sites are prone to all sorts of errors. The papers reporting critical or difficult comments might well be deserving these comments, a few bad examples/comments might skew the average too much when there is a small pool of respondents, and it might be that the ones who have had a bad experience are much more likely to give a review than  those who have had an efficient and good process. In addition, inefficient processing in the past might not be the case anymore as editors and practices also change over time.

Nevertheless, there definitely seems to be a need for some more systematic information about the peer review processes. The more people actually submit their experiences, the more informative the site can be. Of course, this kind of information cannot be taken as being unquestionably trustworthy and one should not see themselves blind to the average scores, but one can presume that if a journal should have a clear dominance of comments that the review process takes well  over a year – this might be an indication of the process being somewhat slow. So, there is some potential to the idea, especially for those who are dependent on having their articles through a peer review process in a reasonable amount of time.

Here are the journals in the area of education. As you can perhaps see, the list is not very comprehensive as of now. However, taking the time to provide information there might in the long run provide relevant information for all, so if this is to work this requires a bit of community input. What do you think? What would be an alternative means for providing this kind of information?