TRIPOD+AI Compliance Checker
Audit prediction model and clinical AI manuscripts against the TRIPOD+AI (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis + AI extension) 27-item checklist.
Workflow
- Read the full manuscript
- Identify study phase: Development (D), Evaluation (E), or Both (D;E)
- Identify modelling approach: regression, machine learning, deep learning, ensemble
- Walk through each item; note applicability column (D, E, or D;E)
- For each applicable item, assign: Reported / Partial / Missing / N/A
- Quote the relevant manuscript text as evidence
- Output a compliance summary + actionable fixes
TRIPOD+AI Checklist (27 Items)
Title and Abstract
| # | Applies | Topic | Requirement |
|---|
| 1 | D;E | Title | Identify as developing/evaluating a prediction model; specify target population, outcome, and modelling approach (regression vs ML) |
| 2 | D;E | Abstract | Structured summary following TRIPOD+AI for Abstracts |
Introduction
| # | Applies | Topic | Requirement |
|---|
| 3a | D;E | Healthcare context | Explain diagnostic/prognostic setting; rationale for model; reference existing models |
| 3b | D;E | Target population | Describe intended population, where in care pathway, who are intended users |
| 3c | D;E | Health inequalities | Describe known health inequalities across demographic/socioeconomic groups; address fairness |
| 4 | D;E | Objectives | State objectives; specify whether development, evaluation, or both |
Methods — Data and Participants
| # | Applies | Topic | Requirement |
|---|
| 5a | D;E | Data sources | Describe source(s) of data; justify selection; assess representativeness |
| 5b | D;E | Data dates | Start/end dates of participant accrual; end of follow-up for prognostic models |
| 6a | D;E | Setting | Study setting (primary/secondary care, general population); number and location of centres |
| 6b | D;E | Eligibility | Inclusion and exclusion criteria |
| 6c | D;E | Treatments | Treatments received; how handled during development/evaluation |
Methods — Data Preparation and Outcome
| # | Applies | Topic | Requirement |
|---|
| 7 | D;E | Data preparation | All preprocessing, cleaning, harmonisation steps; quality checks; consistency across demographic groups |
| 8a | D;E | Outcome definition | Define predicted outcome; time horizon for prognostic models; assessment methods; consistency across subgroups |
| 8b | D;E | Outcome assessors | For subjective outcomes: assessor qualifications and demographics |
| 8c | D;E | Outcome blinding | Whether outcome assessment was blinded to predictor information |
Methods — Predictors
| # | Applies | Topic | Requirement |
|---|
| 9a | D | Predictor selection | Describe and justify initial predictor choice and pre-selection |
| 9b | D;E | Predictor definition | Define all predictors; how and when measured; blinding procedures |
| 9c | D;E | Predictor assessors | For subjective predictors: assessor credentials and demographics |
Methods — Sample Size and Missing Data
| # | Applies | Topic | Requirement |
|---|
| 10 | D;E | Sample size | How determined; justify sufficiency; include calculation details |
| 11 | D;E | Missing data | Approach to missing data with justification |
Methods — Analytical Approaches
| # | Applies | Topic | Requirement |
|---|
| 12a | D | Data partitioning | How data allocated to development/evaluation; partitioning strategy |
| 12b | D | Predictor handling | How predictors handled (functional forms, transformations, standardisation) |
| 12c | D | Model building | Model type with rationale. For ML: architecture, hyperparameter tuning, training procedures. Internal validation method |
| 12d | D;E | Heterogeneity | How variability across clusters (hospitals, countries) was handled |
| 12e | D;E | Performance evaluation | Discrimination (c-statistic/AUC), calibration methods, clinical utility; model comparison if applicable |
| 12f | E | Model updating | Recalibration or updating approaches |
| 12g | E | Prediction calculation | How predictions generated; formula, code, or API details |
Methods — Class Imbalance and Fairness
| # | Applies | Topic | Requirement |
|---|
| 13 | D;E | Class imbalance | Whether imbalance methods used, why, implementation, recalibration steps |
| 14 | D;E | Fairness assessment | Approaches to assess and address fairness across demographic groups |
Methods — Model Specifications and Ethics
| # | Applies | Topic | Requirement |
|---|
| 15 | D | Model output | Output type (probabilities vs classifications); classification thresholds and rationale |
| 16 | D;E | Dev vs eval differences | Differences between development and evaluation in settings, eligibility, outcome, predictors |
| 17 | D;E | Ethical approval | IRB/ethics committee; consent procedures or waiver |
Open Science
| # | Applies | Topic | Requirement |
|---|
| 18a | D;E | Funding | Funding sources and funder role |
| 18b | D;E | Conflicts | All author disclosures |
| 18c | D;E | Protocol | Where protocol accessible; or state not prepared |
| 18d | D;E | Registration | Registry name and number; or state not registered |
| 18e | D;E | Data sharing | Data availability; access restrictions and terms |
| 18f | D;E | Code sharing | Analytical code availability; access conditions |
Patient and Public Involvement
| # | Applies | Topic | Requirement |
|---|
| 19 | D;E | PPI | Patient/public involvement in design, conduct, reporting; or state none |
Results
| # | Applies | Topic | Requirement |
|---|
| 20a | D;E | Participant flow | Flow of participants; outcome event counts; follow-up time; flow diagram recommended |
| 20b | D;E | Participant characteristics | Demographics and key characteristics overall and per setting; predictor values, treatments, sample size, events, missing data; differences across demographic groups |
| 20c | E | Data comparison | Compare predictor distributions between evaluation and development datasets |
| 21 | D;E | Participant counts | Participants and events for each analysis phase (development, tuning, evaluation) |
| 22 | D | Full model specification | Complete model details for reproduction: regression coefficients/intercept, or model code/object/API |
| 23a | D;E | Performance | Performance measures with CIs; subgroup results; calibration plots |
| 23b | D;E | Heterogeneity results | Performance variation across clusters |
| 24 | E | Model updating results | Updated model and its performance |
Discussion
| # | Applies | Topic | Requirement |
|---|
| 25 | D;E | Interpretation | Overall interpretation; fairness considerations; comparison to existing models |
| 26 | D;E | Limitations | Non-representativeness, sample size, overfitting, missing data, measurement bias, generalisability |
| 27a | D | Poor quality input | How model handles poor quality, missing, or out-of-range input data at deployment |
| 27b | D | User requirements | Level of user interaction needed; expertise required |
| 27c | D;E | Future research | Next steps: external validation, implementation, generalisability studies |
ML/AI-Specific Emphasis
These items have expanded requirements for ML/AI models compared to traditional regression:
| Item | ML/AI Extra Requirements |
|---|
| 7 (Data preparation) | Feature engineering, data augmentation, normalisation pipelines |
| 12c (Model building) | Full architecture spec, hyperparameter search space, training/validation split, early stopping, regularisation |
| 13 (Class imbalance) | SMOTE, oversampling, undersampling, cost-sensitive learning |
| 14 (Fairness) | Algorithmic fairness metrics across demographic groups (new in TRIPOD+AI) |
| 3c (Health inequalities) | Equity considerations for model deployment (new in TRIPOD+AI) |
| 18e-f (Open science) | Model weights, training code, inference API sharing |
| 22 (Model specification) | Model weights/code/API, not just coefficients |
Common TRIPOD+AI Gaps
| Frequently Missing | Fix |
|---|
| Item 3c (Health inequalities) | Add paragraph on known demographic disparities in the prediction problem |
| Item 12c (Full ML pipeline) | Document architecture, hyperparameters, training procedure, validation strategy |
| Item 14 (Fairness) | Report model performance stratified by sex, age, race/ethnicity |
| Item 22 (Model specification) | Share model code/weights via GitHub or provide formula with all coefficients |
| Item 18e-f (Data/code sharing) | Publish code on GitHub; share de-identified data or explain restrictions |
| Item 19 (PPI) | State whether patients/public were involved; if not, say so explicitly |
| Item 10 (Sample size) | Use Riley et al. criteria for prediction model sample size |
Output Format
TRIPOD+AI Compliance Report
Study phase: [Development / Evaluation / Both]
Modelling approach: [Regression / ML / Deep Learning / Ensemble]
Manuscript: [filename]
Summary: X/27 Reported | Y Partial | Z Missing | W N/A
(Items assessed based on study phase: D-only / E-only / D;E)
ML/AI-SPECIFIC GAPS:
[Item #] [Topic] — [What's needed for ML/AI compliance]
OTHER MISSING:
[Item #] [Topic] — [What's needed]
PARTIAL ITEMS:
[Item #] [Topic] — [What's present] → [What's missing]
Open science:
Code sharing: [Available (URL) / Not available / Not stated]
Data sharing: [Available (URL) / Not available / Not stated]
Registration: [Registered (ID) / Not registered / Not stated]
Extensions
- TRIPOD-LLM (2024, Nature Medicine): Extension for studies using large language models in biomedical/healthcare. Adds 19 items covering explainability, transparency, human oversight, and task-specific LLM considerations.
- PROBAST (Prediction model Risk Of Bias ASsessment Tool): Companion tool for assessing risk of bias; use alongside TRIPOD+AI for quality appraisal.
Related Skills
/manuscript — Overall manuscript writing and anti-pattern scanning/strobe-check — If the prediction model is developed from an observational cohort, also run STROBE
