Ask Mallard
Core concept

Choosing Bayesian or frequentist inference

4 min read

Bayesian inference gives probabilities conditional on a prior and model. Frequentist inference describes procedures through their behavior across repeated samples.

Fit

When it matters, and when it does not

It matters when

  • You must decide whether a posterior probability or a frequentist interval and test will answer the research question.
  • Earlier evidence may inform a prior, and its relevance and influence need explicit scrutiny.

It matters less when

  • The model ignores clustering, missing data or confounding. Address those design problems first; either inference framework can give misleading answers. Read: The estimand
  • The aim is to obtain a favorable conclusion after seeing results. Follow the prespecified analysis and label any reanalysis. Read: Prespecification and the analysis plan

Before you start

What to check in your study

Decide these first
  1. The outcome and its summary measureoutcomeTypesummaryMeasureSuch as an odds ratio for a yes-or-no outcome. The prior is set on the same scale.
  2. The main prior, and where it comes fromEarlier trials, expert opinion or a deliberately skeptical choice. Write down its center and spread before seeing the data.
  3. Other priors for a sensitivity analysisAt least a skeptical prior centered on no effect and a vague one. The range of answers across them is the honest result.
  4. The effect that matters, and how sure you must beSuch as a 90 in 100 chance that the risk falls by at least 2 in 100. Fix both in advance.
  5. For a trial, how its decision rule behavesSimulate how often it would declare benefit when there is none, and how often it would miss a real effect.
It rests on these
  1. The model for the data is right, just as in any other analysis.How to check: A prior cannot fix a wrong outcome model, missing confounders or ignored clustering.
  2. Evidence borrowed into the prior applies to these patients.How to check: Explain why earlier trials match this population, treatment and outcome. Give them less weight if they may not.
  3. The computer's sampling has settled on the posterior.How to check: Most models are fitted by simulation, called MCMC. Check that several chains agree before reading any result.

Plain language

How it works

Bayesian inference combines a stated prior with the data through a model. Its posterior describes uncertainty conditional on those choices. Frequentist inference evaluates a procedure across repeated samples under stated assumptions. It can provide confidence intervals and tests without assigning a probability to a fixed effect.

How the prior shapes the posterior
Two panels with the same data. With a skeptical prior centered low, the posterior lies between the prior and the data. With an enthusiastic prior centered high, the posterior again lies between them but in a different place.A skeptical priorEffectPriorDataPosteriorAn enthusiastic priorEffectPriorDataPosteriorThe same data give different posteriors under different priors, which is why a credible analysis shows thesensitivity to the prior rather than one chosen posterior.
Conceptual diagram. It shows the structure of the idea, not data from any study.
Read the interval in its own framework

A 95% credible interval holds 95% posterior probability given the prior, model and data. A 95% confidence procedure covers the fixed true effect in 95% of repeated samples under its assumptions. A P value is not the probability of benefit. Both approaches require a suitable model and honest sensitivity checks. Prespecify priors, decision thresholds and trial operating characteristics rather than choosing a favorable framework afterwards.

Worked example

A worked example

Invented for illustration
Setting
A small randomized trial in 200 adults with severe pneumonia, 100 given a new treatment and 100 usual care.
Question
How likely is it that the new treatment lowers death within 30 days?
Approach
A Bayesian analysis of the risk ratio, on the log scale with a normal approximation. Two priors were fixed in advance: a vague one, and a skeptical one centered on no effect and carrying as much weight as the trial.
Result
24 of 100 died with the new treatment and 30 of 100 with usual care: a risk difference of -6 per 100 and a risk ratio of 0.8.
What it means
Under the vague prior, the probability that the treatment lowers deaths is about 83 in 100. Under the skeptical prior, it is about 75 in 100. Neither is a P value, and neither settles the question.

In practice

Recent studies PubMed search · not vetted

Searching PubMed…

Watch out

Common mistakes

The one to remember

The prior is a modeling choice with real consequences, and a Bayesian analysis is only as credible as its justification and the sensitivity analysis around it: reporting one posterior from one prior chosen without discussion is not more honest than a p-value, it is less. A credible interval is also not a confidence interval renamed. And Bayesian methods are not a way to rescue a null trial: computing a high probability of benefit from an enthusiastic prior after the fact is advocacy, not analysis.

  • Reading a P value as the probability that the treatment works.Instead: Only a posterior probability answers that question, and it depends on the prior.
  • Calling a flat prior non-informative.Instead: A prior flat on one scale is not flat on another, and it can favor implausibly large effects. Use a weakly informative prior and say why.
  • Assuming a Bayesian trial needs no control of false positives.Instead: Regulators still ask how often the decision rule would declare benefit wrongly. Estimate it by simulation before the trial starts.
  • Reporting only the probability of any benefit.Instead: Also give the probability of a benefit large enough to matter, and of harm. A tiny effect can be very probable and still unimportant.

Inside Mallard

How Mallard uses this

For a causal question with a binary outcome and an odds ratio, Mallard may offer Bayesian logistic regression. Its method link opens the inference concept; the concept does not fit a model.

Code example in Mallard: Bayesian logistic regression with stated priors (bayesian-logistic-regression).
Atlas node concept.choosing_bayesian_frequentist, the id a plan cue records.
Code for this method
Loading…

Related Learn pages

Related designs and methods

Evidence

Sources & further reading

  1. Bayesian methodology for the design and interpretation of clinical trials in critical care medicine: a primer for clinicians (opens in a new tab) · Crit Care Med · doi 10.1097/CCM.0000000000000576
  2. A Gentle Introduction to the Comparison Between Null Hypothesis Testing and Bayesian Analysis: Reanalysis of Two Randomized Controlled Trials (opens in a new tab) · J Med Internet Res · doi 10.2196/10873
PubMed records checked on 2026-11-07: record verified, confirm it supports this use. Record matching does not establish methodological approval. Examples are invented for illustration.
Last source review: 2026-11-07
AI-generated educational content
Mallard uses reputable methodological and reporting sources, but this material may contain errors or omit important context. Verify important methodological decisions with the cited sources and, when appropriate, a qualified research methodologist or statistician.