Strata Academy

JBI critical appraisal tools explained – observational study checklists

Choosing the right JBI checklist for case series, cross-sectional studies, cohorts, and qualitative research

• There is no single 'JBI score' — each study design has its own checklist.
• Wrong checklist choice (e.g. cross-sectional tool on a cohort) invalidates appraisal.
• JBI suits descriptive observational, prevalence, case series, and qualitative work.
• Intervention studies with randomisation → ROB 2; non-randomised interventions → ROBINS-I.
• Answer yes / no / unclear / N/A per item; summarise limitations in words, not invented numbers.

1. What is JBI?

The Joanna Briggs Institute (JBI) publishes critical appraisal tools for diverse study designs – analytical observational studies, prevalence studies, case series, case reports, qualitative research, and economic evaluations.

JBI methodology is widely taught in nursing, allied health, public health, and mixed-methods programmes across UK universities. Each checklist is design-specific; there is no single universal 'JBI score' applicable to every paper type.

JBI tools emphasise whether a study is trustworthy for its stated purpose – describing prevalence, generating hypotheses, or exploring lived experience – rather than forcing causal language inappropriate to the design.

Official checklists are updated periodically. Use the version aligned with your course or the JBI manual current at appraisal time, and cite the checklist name in your appendix. The JBI Critical Appraisal Tools site lists current PDFs.

For medical students on intercalated degrees or MSc public health modules, JBI often sits alongside CASP checklists. Both teach similar skills with different wording — pick one framework per paper and apply it consistently.

JBI also publishes tools for systematic reviews (JBI SUMARI) — distinct from primary study critical appraisal. Confusing the two in a dissertation methods section is a common marker comment.

Economic evaluations and prevalence studies with analytical components each have dedicated JBI checklists. Do not default to the cohort checklist when the paper estimates prevalence with association testing.

Mixed-methods papers: if qualitative and quantitative components address different questions, appraise each with the matching JBI tool and integrate limitations separately in discussion — do not merge into one quality narrative.

2. Choosing the correct JBI checklist

Match the checklist to the study design described in methods – not the authors' preferred terminology or journal section headings.

If authors call a study 'prospective observational' but randomly assign treatment, the design is an RCT – use ROB 2, not JBI cohort tools.

Diagnostic accuracy studies should use QUADAS-2 with STARD reporting guidance, not generic observational JBI checklists, because validity and applicability domains differ.

When multiple designs appear (e.g. baseline cross-sectional survey plus follow-up), appraise the component relevant to your research question or acknowledge dual-design complexity explicitly.

Audit studies, quality improvement projects, and service evaluations may not fit standard JBI tools — check whether your module expects JBI, CASP, or a bespoke NHS evaluation framework instead.

Note: Using a cross-sectional checklist on a cohort study invalidates your appraisal from the start.

3. Typical JBI domains

While items differ by checklist, JBI tools commonly assess whether the study design can answer the question posed, whether sampling was appropriate, and whether measurement of exposures and outcomes was valid and reliable.

Selection bias items ask whether participants represent the target population and whether recruitment could skew findings. Convenience samples from single NHS trusts limit generalisability even if analysis is statistically sophisticated.

Measurement bias items examine whether exposure and outcome were defined consistently and measured with instruments that were validated for the population studied. A questionnaire validated in US undergraduates may not transfer to UK older adults.

Confounding and analysis items ask whether statistical methods match the design – e.g. logistic regression assumptions for cross-sectional prevalence, adequate follow-up for cohort studies, and appropriate handling of missing data.

Response rates below 60% in survey-based cross-sectional studies often threaten selection validity — note this when scoring representativeness items even if authors report complex weighting.

• Appropriateness of design for the question
• Sample and setting – representativeness and recruitment
• Exposure / intervention / phenomenon measured validly
• Outcome / finding measured with acceptable reliability
• Confounding identified and addressed where possible
• Follow-up adequate for cohort designs
• Statistical analysis appropriate to design and data type

Tip: Selection bias and measurement bias items often determine whether you trust the findings.

4. JBI for qualitative studies

The JBI qualitative checklist assesses congruity between methodology and research question, sampling strategy, data collection, analysis, and presentation of findings with participant voice.

Items on reflexivity ask whether researchers acknowledged their position and influence on data collection and interpretation. This is essential for interview and focus group studies in health services research.

Look for audit trail, member checking, or peer debriefing where appropriate — not every qualitative tradition requires the same rigour markers, but JBI gives a structured starting point for appraisal.

Do not apply quantitative observational JBI checklists to phenomenology or grounded theory papers. Qualitative-specific criteria include theoretical framework clarity and whether conclusions exceed the data.

• Is there congruity between methodology and stated philosophical position?
• Is sampling deliberate and justified (not convenience-only without acknowledgment)?
• Are findings illustrated with participant quotations?
• Is reflexivity addressed — researcher bias and context?
• Are conclusions supported by the analysis presented?

5. JBI vs ROB 2 / ROBINS-I / NOS

Intervention studies with randomisation → ROB 2, not JBI. Non-randomised intervention comparisons with causal intent → ROBINS-I is Cochrane's preferred risk-of-bias framework.

JBI excels for descriptive observational designs, prevalence estimates, case series, and qualitative work where the question is not primarily causal effect estimation.

Newcastle–Ottawa Scale (NOS) remains common in meta-analyses of cohort and case–control studies. Some fields expect NOS stars; others expect ROBINS-I – follow your supervisor and journal convention.

CASP checklists overlap pedagogically with JBI for teaching critical appraisal skills but use different wording – pick one framework per paper and apply consistently. Mixing CASP and JBI item labels on the same worksheet confuses examiners.

6. Answering JBI items

JBI checklists typically use yes / no / unclear / not applicable. 'Unclear' usually means not reported – document as a reporting limitation rather than assuming the method was adequate.

Summarise overall quality in words (major limitations in selection and measurement) rather than inventing a numeric score unless your course explicitly requires one. Some programmes count 'yes' responses — follow local guidance.

Quote short evidence from the paper for each item – page and section – so examiners can verify your judgement. 'Item 5: No — exposure measured by single self-report question without validation (p. 4)' is ideal.

For qualitative studies, assess reflexivity, congruity between methodology and analysis, and whether findings are supported by participant quotations. 'Unclear' on reflexivity is common and should be noted explicitly.

1. Identify true study design from methods, not title.
2. Download the matching JBI checklist PDF.
3. Work item by item with page references.
4. Summarise major limitations by domain (selection, measurement, analysis).
5. State whether findings are trustworthy for the paper's stated aim — not for causal claims beyond design.

7. Worked example – cross-sectional prevalence study

Apply the JBI cross-sectional or prevalence checklist to a published survey. The Health Survey for England methodology papers illustrate sampling frames, weighting, and validated instruments — useful for scoring selection and measurement items.

When appraising any cross-sectional paper, ask first: is the research question descriptive (prevalence, associations) or implicitly causal? JBI suits the former; causal language requires caution and often ROBINS-I if interventions are compared.

8. JBI in UK coursework and dissertations

Many UK nursing, midwifery, physiotherapy, and public health programmes specify JBI as the required appraisal framework. Medical school SSCs and intercalated projects may accept JBI or CASP — check your module handbook before starting.

When writing your appraisal appendix, include the checklist name and version date in the header. Examiners reject generic 'critical appraisal' sections that do not cite which tool was used.

For mixed-methods dissertations, appraise quantitative and qualitative components separately with the matching JBI checklists. A single overall 'quality score' across methods is not methodologically sound.

Integration in discussion: after JBI appraisal, state explicitly whether findings are fit for purpose. A case series with major selection bias may still support hypothesis generation but not safety claims or prevalence estimates.

If your paper is a service evaluation or audit rather than research, JBI may not apply — NHS Health Research Authority guidance distinguishes audit from research. Do not force JBI onto QI reports without supervisor agreement.

Case reports score with the JBI case report checklist — only eight items, but students often overclaim generalisability from n=1. State explicitly that findings are hypothesis-generating.

• Cite checklist name, version, and URL in appendix
• One checklist per study — no hybrid CASP/JBI item lists
• Quote page numbers for each yes/no/unclear judgement
• Separate appraisal for each component in mixed-methods work
• Link limitations to whether the study aim remains credible

10. Journal club checklist (JBI)

Open with design classification in one sentence: 'This is an analytical cross-sectional study of X in Y setting.' Only then name the JBI checklist. Journal club leads lose the room when they announce the wrong tool.

Work through selection and measurement items first — they usually determine whether findings are credible. Save statistical detail for discussion unless analysis items are clearly inappropriate.

End with a fit-for-purpose statement: 'Trustworthy for prevalence estimation but not for causal claims about treatment.' That framing shows you understand JBI philosophy.

If the paper compares two treatments without randomisation, stop and discuss whether ROBINS-I should have been used instead — even if the module specified JBI, acknowledge the limitation.

Prevalence studies with multivariable associations: remind the room that cross-sectional ORs do not establish temporality. JBI may score analysis items as adequate while causal language in the discussion remains inappropriate.

• State design before checklist name
• Prioritise selection and measurement items
• Cite page evidence for contested items
• Fit-for-purpose summary in plain language
• Flag framework mismatch when interventions compared

10. Common mistakes

Framework mismatch is the most frequent JBI error in student coursework. These mistakes are easy to avoid once you classify design before opening any checklist.

• Using one JBI checklist for every paper in a journal club regardless of design.
• Applying JBI cohort tools to RCTs because the setting was a hospital ward.
• Ignoring qualitative-specific criteria (reflexivity, theoretical framework, cultural context) on interview studies.
• Treating 'unclear' items as 'yes' to produce a favourable summary.
• Confusing JBI critical appraisal with JBI systematic review methodology — different toolsets.
• Applying JBI to diagnostic test papers instead of QUADAS-2.

12. StrataResearch and JBI

StrataResearch selects JBI-aligned checklists when observational designs such as case series or cross-sectional studies are detected from manuscript structure.

RCTs route to ROB 2; systematic reviews route to AMSTAR 2 and PRISMA; diagnostic accuracy routes to QUADAS-2 – reducing framework mismatch in mixed journal clubs.

Upload via quick analysis to see structured domain output, then compare against your manual JBI worksheet. Use discrepancies to find methods details you missed on first read.

For qualitative uploads, ensure the correct checklist is selected — automated detection may default to cross-sectional if abstract language is ambiguous. Always confirm design from the methods section before accepting routed output.

Teaching tip: practise design classification on five papers from one journal issue before opening any checklist — speed and accuracy here prevent most framework errors in timed OSCE appraisal.

Allied health students: JBI is often mandatory in nursing dissertations — medical students collaborating on mixed-team projects should align on one appraisal framework per primary study early.

Save the checklist PDF you used with its access date — JBI updates items occasionally and examiners may query version control.

Version control matters for reproducible student appraisal.

• Automatic design routing reduces wrong-checklist errors
• JBI-aligned domains for observational primary studies
• Side-by-side comparison with ROB 2/ROBINS-I when design is ambiguous

Frequently asked questions

What is JBI critical appraisal?

JBI (Joanna Briggs Institute) publishes design-specific checklists to assess whether observational and qualitative studies are trustworthy for their stated purpose. Each design — cohort, cross-sectional, case series, qualitative — has its own tool.

Which JBI checklist should I use?

Match the checklist to the actual study design in the methods section. Cross-sectional surveys use the cross-sectional or prevalence checklist; cohort studies with follow-up use the cohort checklist; interviews use the qualitative checklist.

Can I use JBI for randomised trials?

No. RCTs should be appraised with ROB 2 for risk of bias and CONSORT for reporting. JBI is for non-randomised observational and qualitative designs.

How do I score JBI checklists?

Use yes / no / unclear / not applicable per item. Summarise limitations in prose unless your course requires counting 'yes' responses. Always cite page evidence for each judgement.

JBI vs CASP — which should I pick?

Both teach critical appraisal for similar designs. Follow your module handbook. For a single paper, pick one tool and apply it consistently — do not mix item wording from both on the same worksheet.

Interactive walkthroughs and quizzes load when JavaScript is enabled — the checklist and tables above are fully readable without it.