Strata Academy

Missing Data in Clinical Studies: Attrition, ITT & Imputation

MCAR, MAR, MNAR; complete-case analysis; multiple imputation; and ROB 2 / ROBINS-I domains for attrition bias

• Missing not at random (MNAR) is the hardest problem — sensitivity analyses matter.
• Complete-case analysis is rarely adequate as the sole primary analysis in RCTs.
• Compare baseline characteristics of completers vs non-completers.
• Last-observation-carried-forward is usually inadequate for primary outcomes.

1. Types of missingness

Missing completely at random (MCAR): loss is unrelated to observed or unobserved data — rare in clinical studies. Missing at random (MAR): loss relates to observed variables but not unobserved outcome values after conditioning on observed data — multiple imputation assumes MAR.

Missing not at random (MNAR): loss depends on unobserved values, including the outcome itself — e.g. participants with depression drop out because symptoms worsen. MNAR requires explicit sensitivity analyses; ignoring it produces optimistic or pessimistic estimates depending on direction.

Dropout in trials often relates to side effects, lack of improvement, or death — so complete-case analysis frequently biases results toward the null or away from it depending on which arm loses more participants.

2. Common analysis approaches

Complete-case analysis (available-case) discards participants with any missing values — simple but reduces power and can bias estimates if missingness differs between arms.

Multiple imputation creates several plausible completed datasets using observed relationships, analyses each, and pools results (Rubin's rules). Generally preferred for primary ITT analyses when MAR is plausible and methods are pre-specified.

Last-observation-carried-forward (LOCF) and single-value imputation treat missing values as if known — generally discouraged for primary analyses in contemporary trial reporting except in sensitivity analyses.

Pattern-mixture models and tipping-point analyses explore MNAR scenarios — look for these when dropout is substantial.

• Primary analysis method should be pre-specified in protocol/registry
• Report numbers analysed vs randomised at each stage (CONSORT flow)
• Distinguish missing outcome data from missing covariates in regression
• Per-protocol analysis is secondary — not a substitute for ITT with missing data

Note: A trial with 25% missing primary outcome data and no sensitivity analysis warrants ROB 2 Domain 3 scrutiny even if p < 0.05 in complete cases.

3. Intention-to-treat and missing outcomes

ITT analyses all participants in assigned groups regardless of adherence — preserving randomisation balance. Missing outcomes break ITT unless handled with appropriate methods.

Modified ITT (e.g. 'received at least one dose') deviates from strict ITT — must be labelled and usually secondary. Appraisal should ask whether deviations relate to prognosis.

In observational cohorts, loss to follow-up plays an analogous role to trial dropout — compare characteristics of those lost vs retained.

Tip: Ask: 'If missing participants had worse outcomes, would the conclusion change?' If yes, demand sensitivity analysis.

4. Risk-of-bias frameworks

ROB 2 Domain 3 judges bias due to missing outcome data: availability of data, whether missingness could depend on true outcome values, and appropriateness of analysis.

ROBINS-I includes missing data in several domains, especially for long follow-up cohorts with differential loss.

When appraising systematic reviews, check whether review authors contacted primary study authors for missing data — AMSTAR 2 includes items on data availability.

• Report participant flow from allocation to analysis
• Compare baseline characteristics of completers vs non-completers by arm
• ITT with appropriate missing-data methods preferred for RCT primary analysis
• State reasons for missingness where known (withdrawal, death, loss to follow-up)

5. Appraisal questions for students

Use these when reading results tables or StrataResearch statistical feedback on missing data handling.

1. What percentage is missing in each arm?
2. Is missingness differential between arms?
3. What analysis was primary — complete case, MI, or other?
4. Was the method pre-specified?
5. Are sensitivity analyses reported for MNAR?
6. Does the discussion acknowledge attrition limitations?

6. Sensitivity analyses and tipping points

Sensitivity analyses test whether conclusions hold under alternative assumptions about missing data. Common approaches: best-case/worst-case imputation, varying imputation models, and tipping-point analyses that identify how many missing outcomes would need to favour one arm to nullify the result.

Regulatory and Cochrane guidance expects pre-specified sensitivity plans for trials with non-trivial attrition. Post-hoc sensitivity alone does not rescue a primary analysis that ignored missing data.

In appraisal, ask whether the direction of bias from missing data could explain the observed effect. Differential dropout in the intervention arm because of toxicity may bias toward null or harm depending on outcome definition.

• Best/worst case bounds — useful teaching tool even when authors omit them
• Tipping-point — how extreme would MNAR need to be to change inference?
• Contacting authors for missing participant-level data — standard in Cochrane reviews

7. Worked example: antidepressant trial attrition

Frequently asked questions

Is 10% missing data acceptable in an RCT?

There is no universal threshold. Ten percent may be acceptable if missingness is balanced, reasons documented, and appropriate analysis used. Unbalanced or MNAR-related missingness at 10% can still bias results.

Should I downgrade GRADE certainty for missing data?

Risk of bias due to missing data is one GRADE domain. Substantial loss without adequate handling warrants downgrading for risk of bias in systematic reviews.

When is multiple imputation inappropriate?

When too few observations support the imputation model, when key predictors of missingness were not measured, or when MNAR is likely and no sensitivity analysis is provided.

Does balanced dropout mean no bias?

Balanced proportions do not guarantee MAR. If dropout relates to unobserved worsening in both arms similarly, bias may still occur. Always inspect reasons and completer vs non-completer baselines.

How does StrataResearch flag missing data issues?

Statistical feedback checks ITT reporting, attrition flow, and whether uncertainty (CIs) reflects loss of participants. ROB 2 Domain 3 judgements appear in full manuscript appraisals.

Interactive walkthroughs and quizzes load when JavaScript is enabled — the checklist and tables above are fully readable without it.