Skip to Main Content

Systematic Reviews And Other Review Types

This guide will assist you in starting your systematic, scoping and other reviews.

General Information...

         rayyan logo

Rayyan is a free program designed to assist and speed up the process of conducting a systematic review.

  • It allows multiple reviewers to vote upon uploaded articles for inclusion or exclusion in their study.
  • This voting system can be blinded. 
  • Articles can be uploaded to Rayyan in most popular formats such as Endnote or .RIS.
  • Reviews can be downloaded so they are available offline.
  • Rayyan is also available as a phone app to facilitate access to your review anywhere, anytime. 

For more information about Rayyan please read the below article:

Rayyan — a web and mobile app for systematic reviews.

 

Creating an account

Go to rayyan.ai and select sign up.

Fill out the short form:

         sign up form for Rayyan

 

Creating a new review

Open Rayyan, click on the line that says 'new review' and in the drop down that populates give your new review a title and description. Press 'create'.

                     Rayyan new review

Importing records

Rayyan states it is  compatible with the following text formats: EndNote (.enw), RIS, CSV and PubMed XML.

While you can export records from most databases as .RIS files and import them into Rayyan with ease, I strongly recommend you upload your files into a citation manager such as Refworks or Endnote first.

Benefits of taking this extra step include:

  • A back up copy of all your files located in a separate location.
  • Access to the citation manager's other useful tools such as citation page builder, write and cite et cetera.

Uploading  records to an existing review

1) Open up the review you wish to add records to.

2) On the left, where it says 'search methods' select 'add new'.

additional records to an existing rayyan review

3) Select the additional files you wish to upload and hit 'continue'.

         adding files to a review

Importing records from Refworks

Exporting from Refworks to Rayyan.

1) Select the folder that you wish to export and click on it, in this case I have selected the folder labelled Chris1 and it has 559 article citations in it.

2) On the top menu bar, select the 'share' icon and from the sub menu select 'export references'.

Refworks export

3) On the pop up box that populates, select 'all references' and 'RIS' format. Click on the export button.

Refworks pop up box when exporting folders

4) A new .RIS file will be created. You will be given the opportunity to name it and place it in a location of your choice.

5) Open Rayyan, click on the line that says 'new review' and in the drop down that populates give your new review a title and description. Press 'create'.

                     Rayyan new review        

 

6) Click on 'select files'. Find and select the .RIS file you exported from Refworks, which in this case was labelled Chris1. Press 'continue'.

rayyan select files

7) Your file will upload to your new review. 

Duplicates feature

The detect duplicates feature in Rayyan is one of the best I have seen. However, never presume it does a perfect job of catching all the duplicates. The only way to ensure that is to manually go through your articles.  

To start go to your review.

1) Click on detect duplicates, two blue boxes will populate indicating that the detect duplicate search has begun and how many duplicates were found.

duplicates picture 1

2) Click on  'unresolved'. The unresolved duplicates will now populate as the list of articles. Click on the first article.

detect duplicates 2 rayyan

3) All possible versions of the article which Rayyan suspects as being duplicate will populate. Here we can compare two such suspected articles. The title, authors, journal are all the same and it is safe to say that this is a duplicate. Keep the top article, and click on the 'delete' icon for the second.

duplicates in rayyan 3

4) In the 'possible duplicates' box, located on the top left hand side of your screen, you can now see that there are only 76 articles left unresolved and that 1 has been removed. 

Rayyan duplicate 4

5) Nothing is ever truly deleted on Rayyan, which is a nice failsafe, the article is simply removed from the main review and placed in a duplicate folder.

Blinding

The blind feature will not appear as an option until after you have added other collaborators to the review team. 

To activate or deactivate the blind function you have to be the owner of the review, i.e. the person who originally set up the Rayyan review.

Rayyan's voting system can be blinded or not depending on the reviewers preference. For reasons of bias, I would recommend keeping the blind on. This way when each reviewer selects articles for inclusion/exclusion there can be no bias based on what they are seeing other reviewers select.

Simply click on the blind box to activate or deactivate.

blind on

blind off

Inviting reviewers

Only the review owner (whomever set up the review) can invite other reviewers.

To invite other reviewers to your team, select the review you are working on and press the 'invite' icon.

rayyan invite

A pop up box will populate. Type the email addresses of everyone you wish to invite and select what level you would like them to be set as. 

A collaborator can do everything on the review except delete it, invite others, change the blind status, or delete search methods not created by him/her. 

invite reviewers pop up box

Screening articles

When voting on an article, Rayyan offers three choices:

include exclude maybe

You can either click on the icon --or--in order to speed the process you can simply press the following keys on your keyboard

  • I = include
  • U = undecided
  • E = exclude

If you choose to exclude an article, you can (optionally) select from a preselected list of reasons for the exclusion:

reason for exclusion rayyan

If none of the preselected reasons for exclusion are appropriate, then you can create your own by typing straight into in the reason box. To set it, hover your cursor over  the yellowed new reason and left click.

typing your own reason for exclusion

The new reason will be added to the list of preselected reasons for you and other reviewers to see and use:

creating your own reasons for exclusion

You can also use the label box to include little notes by utilizing the label box. These will show in blue. I would suggest using this to show reasons for inclusion!

label maker in Rayyan

Rayyan & PRISMA

Rayyan will also keep track of exclusion numbers and display them on the left hand side of the screen in the 'exclusion reasons' box. This will be very useful for your PRISMA diagram at manuscript time, i.e. how many articles were tagged with each different reason for exclusion.

While screening is blinded, the numbers shown will be for your screening results only. When blinding is turned off, the total numbers of articles for all screeners will be displayed.

Rayyan and PRISMA

Citing Rayyan

Use the following citation if you used Rayyan in your review:

Mourad Ouzzani, Hossam Hammady, Zbys Fedorowicz, and Ahmed Elmagarmid. Rayyan — a web and mobile app for systematic reviews. Systematic Reviews (2016) 5:210, DOI: 10.1186/s13643-016-0384-4.