Strata Academy

Systematic review methodology – a practical guide for students

From research question and protocol to search, screening, risk of bias, synthesis, and PRISMA reporting

1. Systematic review vs narrative literature review

A narrative literature review selects papers subjectively to support an argument. A systematic review uses explicit, reproducible methods to identify, appraise, and synthesise all relevant evidence for a defined question.

Supervisors and journals expect systematic reviews to minimise selection bias in which studies enter the synthesis. That requires a protocol, comprehensive search, duplicate screening, and documented exclusion reasons.

• Systematic – predefined question, comprehensive search, dual review, synthesis
• Scoping review – maps breadth of evidence; different methods (PRISMA-ScR)
• Rapid review – shortened methods with transparent limitations
• Umbrella review – reviews of existing systematic reviews

2. Formulating the review question (PICO / PEO)

Anchor everything to a clear question. Intervention reviews use PICO: Population, Intervention, Comparator, Outcomes. Prognostic or exposure reviews may use PEO without a comparator.

Define eligibility before searching. Changing criteria after seeing search results introduces bias.

• Population – diagnosis, setting, age, severity; be specific enough to be useful, broad enough to find studies
• Intervention / exposure – dose, duration, delivery
• Comparator – placebo, standard care, active control, or none
• Outcomes – patient-important primary outcome first; secondary outcomes listed a priori
• Study designs – RCTs only, or include non-randomised evidence with separate synthesis?

3. Protocol and registration

Register on PROSPERO, OSF, or a discipline-specific registry before formal screening. The protocol should specify search strategy outline, inclusion criteria, risk-of-bias tools, synthesis plan, and any amendments process.

Protocol deviations happen – document them in the final report with rationale. Undeclared post-hoc changes undermine credibility.

4. Literature search strategy

Systematic reviews aim for sensitivity (find all relevant studies) over precision. Combine controlled vocabulary (MeSH, Emtree) with free-text terms for synonyms and spelling variants.

At minimum, most clinical reviews search MEDLINE (PubMed), Embase, and Cochrane CENTRAL – field-specific databases may be required (CINAHL, PsycINFO, trial registries).

• Work with a librarian early – saves weeks of rework
• Search trial registries (ClinicalTrials.gov, ISRCTN) for unpublished or ongoing studies
• Grey literature – conference abstracts, theses, regulatory documents where relevant
• Reference and citation searching of included studies and key reviews
• Document dates searched and re-run before submission if the review ran >12 months

5. Screening and selection

Import references into Covidence, Rayyan, or similar. Remove duplicates, then title/abstract screening by at least two independent reviewers. Full-text screening follows with exclusion reasons recorded per PRISMA.

Disagreements are resolved by discussion or third reviewer. Kappa statistics can document agreement but are not mandatory in all fields.

6. Data extraction and critical appraisal

Pilot the extraction form on 2–3 papers. Extract population characteristics, interventions, outcomes, effect sizes, and funding. Match each study design to the correct risk-of-bias tool – ROB 2 for RCTs, ROBINS-I for non-randomised interventions, QUADAS-2 for diagnostic studies.

Risk-of-bias assessments should inform synthesis – not sit in a table that readers ignore. Consider sensitivity analyses excluding high-risk studies.

• Extract the same outcome definitions the authors used
• Note unit-of-analysis issues (cluster trials, crossover)
• Record whether intention-to-treat analysis was used

7. Synthesis – narrative and meta-analysis

When studies are too heterogeneous clinically or statistically, narrative synthesis with grouping by design or population may be appropriate. When studies are similar enough, meta-analysis pools effect estimates.

Pre-specify heterogeneity investigation (I², τ², prediction intervals, subgroup analyses). Post-hoc subgroup fishing without multiplicity caution is a common weakness.

8. Reporting with PRISMA 2020

Complete the PRISMA checklist and flow diagram before submission. Report certainty of evidence with GRADE for each important outcome when meta-analysis or qualitative synthesis supports conclusions.

Plain-language summary and implications for practice/research strengthen the discussion – but should not overstate certainty.

9. Where AI tools fit (and where they do not)

AI can assist with search term brainstorming, deduplication support, or drafting plain-language summaries – but dual human screening, risk-of-bias judgement, and GRADE require human accountability.

Generic chat tools do not replace AMSTAR 2 or PRISMA-aligned reporting checks. See our guide on AI and research appraisal for limitations.

10. Practise appraisal on published reviews

Before writing your own review, critically appraise three published systematic reviews in your field using PRISMA + AMSTAR 2. StrataResearch can structure appraisal of review manuscripts you upload – useful for journal club and methodology coursework.

Interactive version (quizzes, walkthroughs) loads when JavaScript is enabled.