Questions to ask when looking for peer review software

Peer review is the standard process used in research and academic conferences, and is essentially the evaluation of materials - whether an abstract submission, a full paper, or even a journal - by other similarly qualified individuals (peers) to assess its validity and significance. This process is vital to ensure editors determine whether the manuscript has been scrutinised, and worthy of being published in a journal, or presented at an academic conference.

The process involves assigning abstracts to reviewers, equipping the reviewer with a reviewing form, and managing, collating and assessing the reviews, so that the follow-up decision stage is streamlined and robust.

Once a laborious admin-heavy task, the peer review process is now usually handled by SAAS abstract management software. Abstract collection is the first part of the process and whilst it is imperative you assess software on its ability to handle this process, you should give equal weight to looking at the features and tools offered throughout the reviewing process.

Obviously, data protection and privacy are a given, but when looking for software to manage your peer review process, we recommend you give some thought to the stages below, and ask yourself, and peer review software providers, the following questions:

Does the reviewing form take care of your peer reviewing requirements?

Before you decide on which peer reviewing software to purchase, ensure you know exactly what information you are going to need from your reviewers. Aside from the usual grades and comments type of questions, do you have additional requirements that you need to collect?

Take a look at the form templates, questions and set ups. Does the system handle single and double blind reviews? What about conflicts of interest? How are they declared and managed? Do you need the reviewer to upload anything - eg. a bio, certification? Can you share reviews directly with submitters, if required? And what if you require a two-stage review process - for example a first review suggesting amendments, followed by a second review of the amended submission?

Peer review software suppliers should be able to talk you through your requirements for collecting peer reviews, and illustrate how their system will be a time-saving, flexible and robust solution.

Is the assigning process simple and user-friendly?

Once your abstracts are in, the next task will be to assign abstracts to reviewers. Good peer review software will make this process simple for you, saving you lots of time and stress.

Will you need to do lots of repetitive tasks - eg, assigning the same submissions to dozens of reviewers? If so, does the peer review software allow for bulk assignment, or ways to automate this process? Will you need to assign submissions based on reviewers areas of expertise? Will you be assigning submissions to reviewers, or the other way round? Does the system allow for both? Can you quickly reassign reviews if a conflict of interest has been declared, or identified.

There is lots to consider when thinking about the assigning process, and you should spend time identifying your must-haves and optionals.

Are the reviewers going to be happy?

Reviewers have a job to do, and making it as easy as possible for them will make your life a lot easier too. When assessing peer review software, ask the following questions:

What does the user interface look like? Is it simple and user friendly? Can reviewers access the system and their assignments in a minimum of clicks? Does the interface reduce the requirement to switch screens - e.g. are the reviewing form and abstracts able to be viewed side-side ? Can reviewers access summaries and reports of their reviews easily and quickly? Do the reviews autosave, which reduces the fear caused by potential power outages and unstable connections?

Keeping reviewers happy is crucial for a successful peer review process as they are more likely to complete their reviews, and less likely to contact you with questions and problems.

How do I keep tabs on reviewers’ progress?

Assigning reviews is one thing, keeping track of who has and hasn’t completed their reviews is another task altogether. Having all the data you need at hand is crucial to manage this process.

Can you see instantly who has completed their reviews? Is there a robust and effective chase-up process built into the system? Can you send reminders and track whether these have been delivered and opened? Can reviewers amend their reviews, if necessary, subject to deadlines? And what if you, as an admin, need to amend a review? And are timestamps collected, just in case issues arise?

Incomplete and late reviews can stall the abstract management process, and sometimes jeopardise the research conference itself, so it’s imperative that the peer review software you decide on can deliver on tracking reviewers’ progress.

There are many peer review software providers that can deal with your abstract management and peer review process. Oxford Abstracts has spent 20 years talking to conference organisers and we’re proud to say we can say a firm ‘yes’ to all the above questions, and a lot more.

If you have any questions, please get in touch. We’d be delighted to show you how our peer review software can handle virtually any submission and review process.