Strata Academy

Study designs explained for critical appraisal

RCTs, observational studies, diagnostics, and evidence syntheses

1. Design determines what you can conclude

Study design is not a marketing label – it defines which biases are most likely, which causal language is justified, and which statistical methods are appropriate.

Before appraising methods, classify the design correctly from the methods section. Misclassification (calling a cohort study an RCT) invalidates the rest of your appraisal and selects the wrong framework.

Authors sometimes use vague terms ('real-world study', 'observational trial'). Read allocation and comparator construction carefully.

StrataResearch automates routing, but examiners expect you to justify design classification manually in coursework.

2. Randomised controlled trials (RCTs)

Participants are randomly allocated to intervention and comparator groups. Randomisation, when implemented with concealed allocation, balances known and unknown confounders in expectation at baseline.

RCTs are the preferred design for causal questions about interventions when ethical and feasible. They are not immune to bias – attrition, lack of blinding for subjective outcomes, and selective reporting remain threats.

Appraise with ROB 2 for risk of bias and CONSORT for reporting transparency. The CONSORT flow diagram shows who was randomised, treated, and analysed.

Intention-to-treat analysis preserves randomisation benefits by analysing participants in assigned groups regardless of adherence. Per-protocol analysis answers a different question and is usually secondary.

• Parallel-group, crossover, cluster, and factorial RCTs each have design-specific considerations
• Cluster RCTs randomise groups (wards, practices) – analyse at correct level
• Crossover trials need washout and carryover assessment
• Reporting: CONSORT flow diagram is essential

3. Observational studies of interventions

Cohort and case–control designs observe exposures without randomisation. Treated and untreated groups may differ in prognosis, care pathways, and measured confounders – confounding is the central threat.

ROBINS-I is the Cochrane-recommended risk-of-bias tool for non-randomised intervention studies. STROBE supports transparent reporting of observational research.

Propensity scores, instrumental variables, and difference-in-differences designs attempt to reduce confounding but require careful ROBINS-I appraisal – matching is not automatic protection.

Observational studies can be the best available evidence when RCTs are unethical or impractical, but causal language should match residual bias risk.

• Cohort: exposure defined → follow-up → outcome (prospective or retrospective)
• Case–control: outcome defined first → look back at exposure (efficient for rare outcomes)
• Interrupted time series: outcomes before/after policy change with comparator
• Cannot alone prove causation; triangulate with RCTs and mechanistic evidence

4. Cross-sectional, case series, and case reports

Cross-sectional studies measure exposure and outcome at one time point – useful for prevalence and association screening but weak for temporal causation.

Case series and case reports describe outcomes in a small number of patients without a comparator. They generate hypotheses for rare conditions or novel presentations.

JBI critical appraisal tools are commonly used for these designs. CASP can support structured reading in journal club teaching.

Do not apply ROB 2 or ROBINS-I to single-arm descriptive studies – there is no comparative intervention contrast to judge.

• Cross-sectional – prevalence, survey data, ecological associations
• Case series – consecutive cases with a defined intervention or exposure
• Case report – single patient narrative; lowest generalisability
• Qualitative designs – interviews, focus groups; separate JBI/CASP qualitative tools

5. Diagnostic accuracy studies

Diagnostic studies compare an index test against a reference standard in the same patients. Validity depends on patient spectrum, test timing, blinding of readers, and whether all patients received verification.

Use QUADAS-2 for risk of bias and applicability. STARD supports reporting of index test, reference standard, and flow of participants.

Spectrum bias arises when only severely ill patients are enrolled – sensitivity and specificity then do not apply to the primary care population you care about.

Sensitivity and specificity alone do not tell you post-test probability without prevalence. Likelihood ratios and predictive values need clinical context.

• Sensitivity, specificity, LR+, LR−, and predictive values
• ROC curves summarise trade-off across thresholds
• Avoid partial verification bias – not everyone gets reference standard
• Index test may be part of reference standard – incorporation bias

6. Systematic reviews and meta-analyses

A systematic review uses explicit, reproducible methods to find, appraise, and synthesise studies addressing a question. A meta-analysis statistically pools estimates when clinical and methodological similarity justify pooling.

Appraise the review process with AMSTAR 2 and ROBIS, reporting with PRISMA 2020, and certainty of evidence with GRADE where authors provide SoF tables.

Primary-study bias and review-process bias are separate problems. A meta-analysis of biased RCTs remains biased – 'garbage in, garbage out'.

Network meta-analysis compares multiple treatments via a common comparator; additional assumptions about transitivity apply.

7. Quick framework chooser

Intervention + randomised allocation → ROB 2 (+ CONSORT reporting). Intervention + not randomised → ROBINS-I (+ STROBE).

Diagnosis → QUADAS-2 (+ STARD). Systematic review → AMSTAR 2 + PRISMA + ROBIS; GRADE for certainty.

Cohort/case–control without intervention comparison → NOS or JBI design-specific checklist. Case series/cross-sectional → JBI.

Use our interactive framework picker on the guides hub if unsure – then read the full framework page.

• RCT → ROB 2
• Non-randomised intervention → ROBINS-I
• Diagnostic accuracy → QUADAS-2
• Systematic review → AMSTAR 2 / PRISMA / ROBIS
• Teaching layer → CASP alongside official tools

Interactive version (quizzes, walkthroughs) loads when JavaScript is enabled.