What does an asymmetric funnel plot mean?

Asymmetry suggests small studies with null or negative findings may be missing — consistent with publication bias. It can also reflect heterogeneity, poor quality in small trials, or chance with few studies. Investigate before concluding.

When should I not run Egger's test?

Cochrane cautions against Egger's test with fewer than 10 studies because of low power and false positives. With very few studies, describe visual inspection and focus on comprehensive search and registry checks.

Does a symmetric funnel plot prove no publication bias?

No. Symmetry is reassuring but not proof. Unpublished studies may be missing in ways that do not produce obvious asymmetry, especially with few data points or homogeneous large trials.

How does publication bias relate to risk of bias in ROB 2?

ROB 2 assesses bias within included trials (randomisation, attrition, etc.). Publication bias is a review-level concern about which studies entered the evidence base. Both can distort the pooled estimate.

Should I include a funnel plot in my dissertation?

Include one for pooled outcomes with enough studies (often ≥10) if your supervisor expects meta-analysis diagnostics. Always pair it with search methods, limitations, and GRADE footnotes — not as a standalone figure.

Strata Academy

Funnel Plots & Publication Bias in Meta-Analysis

Reading asymmetry, Egger and trim-and-fill tests, grey literature, and when funnel plots support GRADE downgrades

Quick answer

A funnel plot graphs each study's effect size against precision (often standard error). Symmetry suggests no strong publication bias; asymmetry is suggestive but not proof — it can also reflect heterogeneity, poor methods in small studies, or chance with few studies. Cochrane treats statistical tests as exploratory and may downgrade GRADE certainty when bias is plausible.

1. What funnel plots show

In a standard funnel plot, each included study is a point: the x-axis is the effect estimate (e.g. log odds ratio, mean difference) and the y-axis is precision (often 1/standard error, so larger studies appear higher).

The intuition: large precise studies should scatter near the pooled effect; small imprecise studies should scatter widely but symmetrically if there is no systematic suppression of negative or null results.

Funnel plots appear in Cochrane reviews, dissertation meta-analyses, and journal submissions after pooling. They are one line of evidence in the publication bias domain — not a standalone quality score.

X-axis: effect size on the chosen scale
Y-axis: precision (1/SE) or sample size
Pooled effect often marked as a vertical reference line
Contour-enhanced funnels add significance regions — use cautiously

2. Interpreting symmetry and asymmetry

Symmetry means small studies are distributed evenly around the pooled estimate. Asymmetry — a gap where negative or null small studies are missing — raises suspicion of publication bias (the file-drawer problem).

Asymmetry has other explanations: clinical heterogeneity (small studies enrolled different patients), methodological differences (small underpowered trials with poor conduct), or selective outcome reporting within included studies.

With fewer than 10 studies, visual asymmetry and statistical tests are both unreliable. Cochrane guidance warns against over-interpreting sparse funnel plots.

3. Statistical tests and sensitivity analyses

Egger's test regresses the standardised effect on precision. A significant result suggests asymmetry — but the test has low power with few studies and can be influenced by heterogeneity.

Begg's rank test and Harbord's test for OR scales are alternatives; all share fragility with small review bodies.

Trim-and-fill imputes missing studies to estimate an adjusted pooled effect. Treat it as a sensitivity analysis exploring 'what if missing studies existed' — not as recovered ground truth.

Egger test — common default; fragile with n < 10
Trim-and-fill — exploratory adjusted pool
PET-PEESE and selection models — advanced; pre-specify if used
Compare fixed and random effects in sensitivity analyses

4. Reducing publication bias before the funnel plot

The best defence against publication bias is review conduct: prospective PROSPERO registration, comprehensive database search, trial registry search (ClinicalTrials.gov, ISRCTN, EU CTR), grey literature, and contact with authors.

Including only published journal articles while excluding registered null trials is a conduct flaw that funnel plots may detect but cannot fix.

AMSTAR 2 critical item 4 expects comprehensive search including grey literature. Appraise search strategy before trusting funnel plot symmetry.

5. Linking funnel plots to GRADE downgrades

GRADE publication bias domain considers funnel asymmetry, missing registered studies, industry sponsorship patterns, and evidence that positive findings are easier to publish in the field.

A downgrade requires plausible bias that would change interpretation — not automatic downgrade for any asymmetry. Footnotes should cite registry comparisons and sensitivity analyses.

When asymmetry is present but trim-and-fill barely shifts the pooled estimate, reviewers may judge bias less serious than when the adjusted effect crosses the null.

Suggestive asymmetry + missing registry studies → stronger downgrade rationale
Symmetric funnel + comprehensive search → less concern, not zero concern
Small review bodies → often 'publication bias not assessed' or very low certainty
Document when tests were not run and why (too few studies)

6. Reading funnel plots alongside forest plots

Forest plots show which studies drive the pooled diamond and how confidence intervals overlap. Funnel plots show whether small studies are missing from one side of the distribution.

If one large trial dominates the forest plot, funnel asymmetry among remaining small studies may have little impact on the pooled estimate — note this in appraisal.

Use our interactive forest plot tool to rehearse how study size relates to weight before interpreting funnel scatter.

7. Appraisal checklist for students

When reviewing a published meta-analysis, ask: Was the search comprehensive enough to detect missing studies? How many studies were pooled? Was asymmetry tested appropriately? Did sensitivity analyses change conclusions?

In your own dissertation, report funnel plots for pooled outcomes with ≥10 studies where feasible, state tests used, and discuss limitations honestly when study counts are low.

Confirm number of included studies — defer formal tests if < 10
Inspect visual asymmetry and note gap location (missing null/negative small studies?)
Cross-check against trial registries and grey literature in methods
Run and report pre-specified sensitivity analyses (trim-and-fill, exclude small studies)
Link findings to GRADE publication bias footnote in SoF table
State whether asymmetry likely changes clinical interpretation

Interactive version (quizzes, walkthroughs) loads when JavaScript is enabled.