We have been pleased to see that our novel referee process on F1000 Research generally seems to be working well (and fast), with many authors now revising their articles (see our first two revisions from Frederick Morfaw and Gavin Oliver). We have made a number of changes to our refereeing process over the past few weeks and I wanted to provide an update to clarify these changes and explain the reasoning behind them.
Adding a referee ‘statement’
One major change you may have noticed is that we are now asking our referees to agree to a clear statement clarifying their suitability as a referee for that article. This means that a reader can be sure that the person who is making the assessment of the article’s scientific quality (which is what ultimately decides whether an article becomes indexed) is someone who is happy to openly declare that they are suitably familiar with the topic in question. This line is included in the referee report and is especially important where the referee has provided an ‘Approved’ status but they do not feel they have anything further to add in terms of a referee report.
We have also gone through all our articles to-date and contacted those referees retrospectively. This proved an interesting exercise in that all referees agreed to this statement, except for one who on further reflection, decided that they did not feel suitably qualified and therefore asked us to withdraw their referee report. This was a request we contemplated carefully; following discussion with the articles’ author, we agreed that it would be inappropriate for such a referee status and report to remain. We also made the decision that any future such amendments to our procedures would only be implemented going forward rather than amending existing indexed articles. As it happened, another referee provided an ‘Approved’ status for that article the same day and hence the article was able to remain in the status of ‘Indexed’. Hence, although this change has not impacted the overall outcome for the article in question, we wanted to clarify what had happened particularly for anyone who had viewed that article previously.
Referee name approval
Since my last post on this subject, we have made a further change to our process for referee name approval. Names submitted by authors still go through an internal check to ensure they have no obvious conflict and that they are of a suitable standing within their community. However, following quite some discussion and feedback from some of our authors, we have stopped asking the Editorial Board to verify referee names. Given that all referee names are out in the open, and that we are requiring each referee to publicly declare their suitability to referee the article in question, it seemed an unnecessary (and in places, an inappropriate and non-transparent) step to ask a single unnamed Editorial Board member to pass judgement on the author’s referee suggestions.
Other referee and commenting changes
Other changes made include the fact that we now require any referee selecting the ‘Approved with Reservations’ or ‘Not Approved’ status to accompany this with a report to explain the concerns they have with the science in the article.
We have also brought the user comments more to the fore so that they are still a level below the referee reports, but they are better recognised as being an important contribution to the article.
We continue to listen to feedback from authors, referees, our Editorial Board and Advisory Panel, and our readers, and we encourage you to continue providing your comments on these changes or on any other aspects of our novel publishing system.