Registration and reporting bias
The two biases
PUBLICATION bias is whole studies not appearing, disproportionately those with null or unfavourable results. OUTCOME REPORTING bias is studies appearing with some of their outcomes omitted or redefined. Both make the visible literature more favourable than the evidence, and a meta-analysis of the visible literature inherits both.
What registration provides
- A public record that the study existed, so a systematic review can find studies that were never published.
- The prespecified primary outcome, checkable against what is eventually reported.
- A date, which is what distinguishes prospective registration from retrospective.
- An amendment history, which shows what changed and when.
- For systematic reviews, PROSPERO serves the same function, and the protocol is registered before the search.
Detecting it in a review
Funnel plot asymmetry and tests for small-study effects can suggest publication bias, though asymmetry has other causes and the tests have little power with few studies. Comparing published reports with registry entries and protocols is more direct. The strongest response is a search that reaches beyond published articles: registries, regulatory documents, conference abstracts and contact with investigators.
Reporting a null result is a contribution
The obligation runs to investigators as well as journals. A completed study whose results are never posted has consumed participants' time and risk and returned nothing, and most registries now accept summary results directly.
What to watch out for
Publication bias and outcome reporting bias are different and the second is harder to see: a trial can be published in full while quietly dropping the outcomes that did not reach significance, and comparisons of protocols with publications find statistically significant outcomes roughly two to five times more likely to be fully reported. A registry entry created or amended after results are known provides none of the protection, so the registration date and the amendment history are part of the evidence.
Clinical example
A systematic review finds eight published trials, all favourable. The registry lists fifteen completed trials of the same intervention. The seven missing trials are not a technicality; the pooled estimate from the eight is a biased sample and should be reported as such.
- Systematic review of the empirical evidence of study publication bias and outcome reporting bias (opens in a new tab) · PLoS One · doi 10.1371/journal.pone.0003081
- Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles (opens in a new tab) · JAMA · doi 10.1001/jama.291.20.2457