Systematic Reviews And Other Review Types

A scoping review is a relatively new approach to evidence synthesis and it differs from systematic reviews in its purpose, but the research methodology remains largely the same. The original established guidelines for a scoping review were established by Arksey and O'Malley, but since that time the format has evolved.

Purpose:

The purpose of a scoping review is to provide an overview of all the available research evidence that is available on a topic. They often present findings in visual tables and graphs.

One of the main uses of a scoping review is to identify any possible gaps in the current literature and justify either a full-fledged systematic review or primary study on a topic such as a randomized controlled trial.

Methodology:

Step 1. The average timeframe of a scoping review can be anywhere between six months and eighteen months. Therefore, before you commit to a scoping review, the larger repositories such as Pubmed, Joanna Briggs, Cochrane, and Prospero should be looked at to see if there is already an existing scoping or systematic review on the subject.

Step 2. Consult with your librarian to find usable keyword synonyms and relevant subject heading terms (keyword searches and subject heading searches produce different results). Isolating the perfect search synonyms lays the foundations for all the work to come--invest time here!

To get the best answers, it is important to ask the best questions! For quantitative reviews use PICO and find synonyms for each category.

Population--what is the health condition you are looking at?

Intervention--what treatment/solution are you investigation?

Comparison--what alternate treatment/solution are you comparing the intervention to. When left blank the default is the intervention as compared to no treatment?

Outcome--from a research perspective, including the outcome to your search string potentially adds bias so it should not be incorporated:

Example:  If you search for the effects of 'brand X' medicine on a population of 'migraine sufferers' and incorporate the outcome 'reduced symptoms'--then you would only get those articles that listed the positive results--you may be missing out on articles that list negative results.

Step3. With these terms, devise an effective search string and determine which databases and other sources will be utilized. Librarians are considered experts in the field of research so the mere act of consulting with your librarian will help the status and prestige of your review.

Step 4. Once you have determined the final selection of keywords, run one last search across the big repositories: Pubmed, Joanna Briggs, Cochrane, and Prospero should be looked at to see if there is already an existing scoping or scoping review on the subject.

Step 5. Decide if you wish to register your scoping review?

In theory, registering your scoping review with a repository such as Prospero (free) plants a flag in the topic that lets others know you are already researching it. In practice, gaps in the literature can be hard to find and new ideas for avenues of research are hot commodities. Registering your review can also advertise your idea. 

Should you decide to register your review, per PRISMA protocols you will need at least one official search strategy from one database for a scoping review and an official search strategy for all databases utilized in the case of a scoping review. Your librarian can help with this.

Step 6.  An exhaustive search of the literature should be undertaken and findings recorded on the recommended PRISMA diagram (see below). Upload the results to a citation manager such as Refworks or Endnote.

On the left-hand side of the diagram, tally up all the results you got from a database search.

On the right-hand side, tally up all the results you got from searching:

• • • Physical journals and books
    • E-journals and books
    • Gray literature (anything scholarly but not published and/or peer-reviewed –e.g. government studies and white papers, student dissertations, conference minutes.

Step 7. Record your search.

Step 8. Prepare an ‘A Priori’ inclusion/exclusion table. Upon what criteria do you wish articles to be excluded or included in your review? (A Priori simply means before knowledge which is why it should preferably be done before any eliminations are made)

Step 9. Once all the articles have been found and collated, a series of elimination rounds should take place per PRISMA protocols and results recorded on the PRISMA diagram below.

Eliminate duplicate articles--this step can be undertaken by an individual.

Eliminate articles based on their title and abstract using your inclusion/exclusion criteria.

• • • This step must be undertaken by a team of at least two (preferably three) people.
    • Reviewer (A) and reviewer (B) go to their separate corners and independently select which articles to include and exclude.
    • The results are compared. 
    • Per Arksey and O'Malley, disagreements in selections can be resolved via conversation. However, this can lead to bias, e.g. if one member of the team has a higher status such as being the lead researcher then he/she may get her way more often than not. A better way to resolve disagreements in selections is to add a third member of the team and have him/her come in and cast a deciding vote on those articles that are in dispute.

Eliminate articles based on the full text using your inclusion/exclusion criteria.

• • • This step must be undertaken by a team of at least two (preferably three) people.
    • Reviewer (A) and reviewer (B) go to their separate corners and independently select which articles to include and exclude.
    • The results are compared. 
    • Per Arksey and O'Malley, disagreements in selections can be resolved via conversation. However, this can lead to bias, e.g. if one member of the team has a higher status such as being the lead researcher then he/she may get her way more often than not. A better way to resolve disagreements in selections is to add a third member of the team and have him/her come in and cast a deciding vote on those articles that are in dispute.

Step 10. What you have left is commonly referred to as the ‘golden standard’ --those articles or book chapters et cetera that are a perfect fit for your review’s topic.

At this point, inspect the citation pages of all the sources in your ‘golden standard’. If the article is perfect for your needs then it stands to reason that some of the articles the author used in their research might also be a perfect fit. Record this number to the existing number in your ‘articles from other sources’ box. Subject these new sources to the same inclusion/exclusion elimination criteria. If you find any that will be added to your ‘golden standard’ then these, in turn, must have their citation pages investigated. This process continues until the citations fall outside your specified date range.

Inspect the ‘cited by’ pages of all the sources in your ‘golden standard’ using a database such as Google Scholar. If the article is perfect for your needs then it stands to reason that if it has been cited by another author then that article might be a good fit for your needs also. Record this number to the existing number in your ‘articles from other sources’ box. Subject these new sources to the same inclusion/exclusion elimination criteria. If you find any that will be added to your ‘golden standard’ then these, in turn, must have their cited by results investigated. This process continues no more ‘cited by’ articles are added to your golden standard.

Step 11. Once you have finished all of this, typically some time will have passed. It is highly recommended that one last search be conducted across the databases to pick up any new articles.

Step 12. Have your librarian check the articles that have made it to your 'golden standard' and subsequently their citations for any possible retractions.  

At this point your Librarian has helped you with everything within his/her scope and leaves the process. Once you are finished though your librarian can help you once again by helping identify potential publishers.

Step 13. Critical appraisal of sources (optional) 

Step 14. Collate, chart, tabulate, and present your findings

Suggested Resources:

Article: Scoping studies: towards a methodological framework -Arksey & O'Malley

Book: Joanna Briggs Institute Chapter 11 Scoping Reviews

Article and video: Cochrane Training, Scoping reviews and how to do them 

[Retractions] is this the biggest problem in science today? -- big think article

What is a meta-analysis?

Meta-analysis is the statistical procedure for combining data from multiple studies. 

In order to conduct a meta-analysis you must first have completed a systematic review. It is highly suggested if you plan to conduct a meta-analysis that your systematic review be limited to those studies that produce quantifiable, numerical data sets, e.g. randomized controlled trials. 

Why perform a meta-analysis?

As different studies will have different variables, such as participant numbers, different standards of quality control, et cetera so too will they show differing results. The results of one randomized controlled trial do not provide definitive proof of an interventions success on a particular health condition.

However, if several randomized controlled trials can be found on the same subject and are similar enough that they can be equated, then they can be put through standardized statistical procedures. Their results can be subsequently combined via such instruments as a forest plot graph. It becomes possible to observe whether or not there is a a statistically significant degree of success for whatever intervention is being looked at on the condition being studied.

While certain reviews such as a narrative review or a systematic review can be used for the purpose of synthesizing the results of multiple studies, these reviews can be subjective (different experts can come to different conclusions). Meta-analysis; by contrast, applies objective statistical formula.

Suggested Resources:

Article: A guide to conducting meta-analysis by Cheung & Vjayakumar 

E-book: eta-analysis : a structural equation modeling approach by Mike W. Cheung

Mixed methods isn't so much a review in and of itself; rather, it is an approach that can be taken with existing review types, e.g. a mixed methods literature review, a mixed methods systematic review. 

Purpose:

Elements of both quantitative and qualitative research are included and merged together via a Mixed-Method Review (MMR).

Merging objective numerical data sets with subjective observational data is a difficult task that has the potential to produce a holistically greater whole or go asunder badly. 

Research Methodology:

Depending on what level of review you are applying the Mixed Methods approach to will determine the need to conduct an exhaustive search or not. For example, a mixed-methods systematic review would require an exhaustive search while a mixed-methods literature review would not,

The best way to integrate the two datasets is ask yourself what question it is you are trying to answer, what element(s) from each technique would serve to enhance that answer, and so you need to fully understand the strengths and weaknesses of each:

Qualitative Research

Quantitative Research

To get the best answers, it is also important to ask the best questions! For the search itself always use PICO as it will pull back the most results, but when writing up your paper:- for quantitative data answer using the PICO format and for qualitative data answer using the SPIDER format.

PICO

Population--what is the health condition you are looking at

Intervention--what treatment/solution are you investigating

Comparison--what alternate treatment/solution are you comparing the intervention to (when left blank the default is as compared to no treatment)

Outcome--what is the outcome of the intervention treatment? Does it work better than the comparison treatment or no treatment?

From a research perspective, including the outcome to your search string adds bias so it should not be incorporated:

Example:  If you search for the effects of 'brand X' medicine on a population of 'migraine sufferers' and incorporate the outcome 'reduced symptoms'--then you would only get those articles that listed the positive results--you may be missing out on articles that list negative results.

SPIDER

Sample--how you choose your participants (randomly or other), where you chose your participants (location), when you choose your participants, and what population you chose to target for your participants (race, gender, age, socio-economic status, etc.) can all affect bias significantly in a qualitative paper. 

Phenomenon of Interest--what health condition is being looked at.

Design--how the study was designed and conducted. 

For example, was your sample selected and put into their control and experimental groups blind, or double-blind?

Evaluation--What tool was employed for measurement of outcome? It might be subjective and not necessarily empirical.

Research--Qualitative, or quantitative, or mixed?

Suggested Resources:

Article. PICO, PICOS,and SPIDER

E-book. Mixed Methods Research: Merging Theory with Practice. 

Purpose:

A literature review also known as a narrative review is a low-level review that provides a comprehensive summary of previous research on a topic. It can utilize and incorporate surveys, scholarly articles, books, and other sources relevant to a particular area of research. 

This type of review should summarize previous studies and clarify the previous research. 

In writing the literature review, the purpose is to convey to the reader (or researcher) what general information there is on a topic, and what the strengths and weaknesses are with regard to that information.

Research Methodology:

There is no strict methodology for a literature review. The researcher can choose which databases or sources to use and how many. The researcher does not have to apply any strict inclusion/exclusion criteria on what to select and include. An exhaustive search to capture all relevant material on a given subject, adhering to a strict methodology, can be utilized but this is not typical.

Suggested Resources:

Health sciences literature review made easy: the matrix method. by Judith Garrard

Preparing literature reviews: quantitative and qualitative approaches by M. Ling. Pan Monica Lopez