The Complete Guide to Systematic Reviews

What Is a Systematic Review?

A systematic review is a rigorous, transparent method for identifying, evaluating, and synthesizing all available evidence on a specific research question. Unlike traditional literature reviews, systematic reviews follow a predefined protocol with explicit inclusion criteria, comprehensive search strategies, and standardized quality assessment.

Systematic reviews are considered the gold standard of evidence synthesis. They form the foundation of evidence-based medicine, clinical guidelines, health policy, and Cochrane Reviews. Their reproducibility and methodological rigor make them the most reliable source of evidence for decision-making.

Key characteristics of a systematic review:

• A clearly formulated, answerable research question
• A predefined, registered protocol (e.g., on PROSPERO)
• A comprehensive, reproducible search strategy across multiple databases
• Explicit inclusion and exclusion criteria
• Independent screening by at least two reviewers
• Standardized data extraction and risk of bias assessment
• Transparent reporting following PRISMA 2020 guidelines

Types of Evidence Synthesis

Not every review is a systematic review. Understanding the differences helps you choose the right approach for your research question.

Systematic Review vs. Meta-Analysis

A systematic review is the overarching process — searching, screening, extracting, and synthesizing. A meta-analysis is an optional statistical step within a systematic review that combines quantitative results into a pooled effect size. Not every systematic review includes a meta-analysis; it depends on whether the studies are similar enough to combine statistically.

Systematic Review vs. Scoping Review

Systematic reviews answer a specific, narrow research question and assess the quality of included studies. Scoping reviews are broader — they map what evidence exists on a topic without necessarily appraising study quality. Scoping reviews follow the PRISMA-ScR extension and JBI methodology.

Systematic Review vs. Literature Review

A literature review (or narrative review) provides a general overview of a topic. It doesn't require a systematic search, predefined inclusion criteria, or quality assessment. While useful for background sections, literature reviews are not considered reliable evidence synthesis because they're prone to selection bias and are not reproducible.

How to Conduct a Systematic Review: Step-by-Step

The following steps align with the Cochrane Handbook and PRISMA 2020 methodology.

1. Define Your Research Question

A well-defined question is the foundation of your review. Use the PICO framework for intervention reviews:

• Population — Who are you studying?
• Intervention — What treatment or exposure?
• Comparison — What is the alternative?
• Outcome — What do you want to measure?

For qualitative or scoping reviews, consider the PEO (Population, Exposure, Outcome) or PCC (Population, Concept, Context) frameworks instead.

2. Write and Register a Protocol

Your protocol defines how you will conduct the review before you start. It should include your research question, search strategy, eligibility criteria, data extraction plan, and analysis methods. Registering your protocol prevents duplication and increases transparency.

• PROSPERO — Free international prospective register for systematic reviews with health-related outcomes
• OSF (Open Science Framework) — Alternative for non-health reviews
• PRISMA-P — Checklist for reporting protocols

3. Develop a Search Strategy

Your search must be comprehensive and reproducible. Search at least two databases relevant to your topic, and document every search string, filter, and date. Consider:

• Using Boolean operators (AND, OR, NOT) and MeSH/Emtree terms
• Searching multiple databases (see the databases section below)
• Including grey literature (dissertations, conference proceedings, preprints)
• Hand-searching reference lists of included studies
• Consulting a research librarian for complex strategies

Report your search strategy using the PRISMA-Search extension for maximum transparency.

4. Screen Studies

Screening happens in two phases: title/abstract screening to remove clearly irrelevant studies, followed by full-text screening to apply your complete eligibility criteria. At least two independent reviewers should screen each study, with conflicts resolved by discussion or a third reviewer.

This is typically the most time-consuming step. For a review with 3,000 search results, manual screening can take 100+ hours.

5. Extract Data

Design a standardized data extraction form and pilot it on 2–3 studies. Two reviewers should extract data independently. Common fields include:

• Study characteristics (authors, year, country, design)
• Participant demographics (sample size, age, sex)
• Intervention and comparator details
• Outcome measures and results (means, SDs, effect sizes, p-values)
• Funding and conflict of interest disclosures

6. Assess Risk of Bias

Every included study must be assessed for methodological quality. Use the appropriate tool based on study design:

• RoB 2 — Cochrane's tool for randomized controlled trials
• ROBINS-I — For non-randomized studies of interventions
• Newcastle-Ottawa Scale (NOS) — For cohort and case-control studies
• JBI Critical Appraisal Checklists — Suite of tools for various study designs

Present risk of bias results in a summary table or traffic-light plot. Assess bias at both the individual study level and across studies (e.g., using funnel plots for publication bias).

7. Synthesize Results

If studies are sufficiently similar, conduct a meta-analysis to produce pooled effect estimates. Key decisions include:

• Effect measure: Mean difference (MD), standardized mean difference (SMD/Hedges' g), odds ratio (OR), risk ratio (RR), or risk difference (RD)
• Pooling model: Fixed-effect (inverse variance) vs. random-effects (DerSimonian-Laird), Mantel-Haenszel, or Peto
• Heterogeneity: Cochran's Q test, I² statistic, tau², prediction intervals
• Publication bias: Funnel plots, Egger's test, trim-and-fill analysis
• Subgroup & sensitivity analyses: Test robustness and explore sources of heterogeneity

If meta-analysis is not appropriate (e.g., too few studies or too much clinical heterogeneity), use a narrative synthesis with structured summary tables and vote counting.

8. Report and Publish

Report your review using the PRISMA 2020 27-item checklist and flow diagram. Key reporting elements:

• PRISMA flow diagram showing study selection at each stage
• Characteristics of included studies table
• Risk of bias summary and assessment
• Forest plots for meta-analyses (if applicable)
• GRADE Summary of Findings table for certainty of evidence
• Registration number (PROSPERO) and protocol deviations

Use the GRADE framework to rate the certainty of evidence for each outcome (high, moderate, low, very low).

Key Frameworks & Reporting Guidelines

These are the standards and tools every systematic reviewer should know.

Systematic Review Software & Tools

Dedicated software helps manage the complexity of systematic reviews. Here are the main options:

Many teams also use reference managers like Zotero, Mendeley, or EndNote for importing and deduplicating references before moving to a dedicated systematic review tool.

Databases for Systematic Review Searching

A comprehensive search requires multiple databases. The choice depends on your discipline and research question.