Strata Academy

ROBINS-I explained – risk of bias for non-randomised intervention studies

Seven bias domains, confounding, when to use ROBINS-I instead of ROB 2, and appraisal tips for cohort and before–after studies

1. What is ROBINS-I?

ROBINS-I (Risk Of Bias In Non-randomised Studies of Interventions) is the standard framework for assessing risk of bias when participants are not randomly assigned to intervention and comparator groups.

Version 2 refines signalling questions and overall judgement algorithms. It is published via riskofbias.info and used in Cochrane reviews when RCT evidence is insufficient or complementary.

ROBINS-I addresses a different threat model than ROB 2: confounding and selection into treatment are usually more important than allocation concealment.

2. When to use ROBINS-I

Use ROBINS-I for comparative studies of interventions where groups were formed by clinician or patient choice, policy roll-out, or other non-random mechanisms.

Typical settings include comparative effectiveness research in electronic health records, before–after policy changes, and cohort studies where treated and untreated patients differ systematically at baseline.

Do not use ROBINS-I for single-arm case series, cross-sectional prevalence surveys, or diagnostic accuracy studies – each has design-specific tools (JBI, QUADAS-2).

If authors claim randomisation, verify allocation concealment and sequence generation before defaulting to ROBINS-I. Quasi-random allocation (alternate days, odd/even charts) is not true randomisation.

• Prospective or retrospective cohort studies comparing treatments
• Before–after studies with a concurrent or historical comparator
• Interrupted time series with intervention introduction
• Studies using propensity scores or instrumental variables – still ROBINS-I, not ROB 2

3. ROBINS-I bias domains (overview)

ROBINS-I organises bias into domains spanning confounding, selection, measurement, and reporting. Work through official signalling questions for the target effect (assignment or starting intervention).

Version 2 distinguishes judgements for different target populations and effect types – read the paper's estimand before scoring. A per-protocol effect and an intention-to-treat effect can have different bias profiles.

Each domain uses signalling questions mapped to algorithms producing low, moderate, serious, or critical risk of bias at domain level. Do not skip to overall judgement without domain worksheets.

Supplementary appendices and protocol registrations often contain detail on exposure definitions and follow-up windows essential for Domains 3–5.

• Bias due to confounding – are intervention and comparator groups comparable on prognostic factors?
• Bias in selection of participants into the study – does the study sample represent the target population?
• Bias in classification of interventions – was exposure measured accurately and consistently?
• Bias due to deviations from intended interventions – co-interventions, adherence, crossover
• Bias due to missing data – loss to follow-up related to outcome or treatment
• Bias in measurement of outcomes – differential ascertainment between groups
• Bias in selection of the reported result – outcome switching, subgroup fishing

4. Confounding – the dominant concern

In RCTs, randomisation balances confounders in expectation. In observational studies, treated patients may be sicker, more motivated, or cared for in different centres – all of which can mimic or mask treatment effects.

Look for statistical adjustment (regression, propensity scores, matching), negative control outcomes, or designs that reduce confounding (e.g. regression discontinuity). Absence of adjustment is not automatically high risk if confounding is implausible – but justify that judgement.

• Were key prognostic variables measured and adjusted?
• Is there a plausible unmeasured confounder that could explain the finding?
• Did authors perform sensitivity analyses for unmeasured confounding?

5. ROBINS-I vs ROB 2 vs NOS

ROB 2 is for randomised designs. ROBINS-I is for non-randomised intervention comparisons. Newcastle–Ottawa Scale (NOS) appraises cohort and case–control studies but is not intervention-specific – Cochrane prefers ROBINS-I for causal intervention questions in observational data.

• Policy evaluation without randomisation → ROBINS-I
• Cohort describing prognosis without intervention comparison → may use NOS or design-specific tools
• RCT mislabelled as cohort → reclassify and use ROB 2

6. Overall risk of bias judgement

As with ROB 2, domains feed an overall judgement of low risk, moderate risk, serious risk, or critical risk of bias (terminology per ROBINS-I v2 materials).

Document which contrast you judged (e.g. drug A vs standard care at 12 months). Different time points can have different bias profiles.

7. Common mistakes

Applying ROB 2 signalling questions to observational papers.

Ignoring immortal time bias in database studies of treatment initiation.

Treating propensity matching as automatic protection against all confounding.

Scoring NOS and ROBINS-I interchangeably on the same paper without clarity.

8. How StrataResearch applies ROBINS-I

When study-type detection identifies a non-randomised intervention design, StrataResearch routes to ROBINS-I-aligned domains rather than ROB 2.

Upload the manuscript via quick analysis to see structured domain output you can compare against the official riskofbias.info workbook.

Pair ROBINS-I appraisal with STROBE reporting checks and our regression essentials guide when authors use adjusted models for confounding.

In systematic reviews, expect reviewers to present ROBINS-I traffic-light plots alongside RCT ROB 2 plots – not a single merged score.

Interactive version (quizzes, walkthroughs) loads when JavaScript is enabled.