Markov model
A Markov model moves a cohort between defined states over repeated cycles, accumulating costs and outcomes under explicit transition assumptions.
Fit
When to use it, and when not to
Use it when
- Risks or events recur, and the time spent in each state matters.
- You can define states, cycle length and transition probabilities for the intended population.
Not when
- The pathway is short and finite. A decision tree may be enough. Read: Decision tree model
- History cannot be represented adequately by your states. Expand them or consider individual simulation. Read: Choosing an economic evaluation
Before you start
What you need
- Mutually exclusive states covering the cohortInclude absorbing states where appropriate and specify initial occupancy.
- Cycle length and horizonAlign probabilities with the cycle length; a rate is not a probability.
- State costs, health values and transition inputsState the perspective, discounting rules and how within-cycle timing is handled.
- Current state and modeled time capture the relevant history.How to check: If time in state changes risk, add duration states or use a richer model.
- Each transition row sums to one.How to check: Check nonnegative probabilities, absorbing states and conservation of the cohort.
- The structure and inputs represent the decision.How to check: Compare outputs with external evidence and explore parameter and structural uncertainty.
Plain language
How it works
Represent the cohort by its occupancy in each state. Multiply occupancy by a transition matrix to obtain the next cycle. A row gives all next-state chances from one current state.
Accumulate state and transition costs or health values over the horizon. A model can allow transitions to change with time, while still needing enough state information about history.
Economic evaluation is not one of the five research areas this hierarchy covers. That describes the hierarchy, not the quality of a model. Sonnenberg and Beck explain repeated-event modeling. The ISPOR-SMDM guidance describes model structure, validation and uncertainty.
Worked example
A worked example
- Setting
- A cohort model tracks stable, unwell and dead states over equal cycles.
- Question
- Where is the cohort after one cycle?
- Approach
- Start with 70 stable, 20 unwell and 10 dead. Transition rows are (0.8, 0.15, 0.05), (0.1, 0.7, 0.2), and (0, 0, 1).
- Result
- Expected occupancy becomes 58 stable, 24.5 unwell and 17.5 dead: 17.5 in 100 in the absorbing state.
- What it means
- Occupancies still total 100. Fractional people are expected cohort counts, not observed individuals.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
A cohort Markov model assumes the present state captures the history needed for future transitions. If duration or prior events matter, expand the states or use another model.
- Using an annual probability in a monthly cycle unchanged.Instead: Convert inputs with a justified rate or time model.
- Letting an absorbing state return to another state.Instead: Keep its probability of remaining there equal to one.
Inside Mallard
How Mallard uses this
Plans do not link to this method yet. Mallard does not build or run economic decision models.
analysis.markov_model, the id a plan cue records.Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Markov models in medical decision making: a practical guide (opens in a new tab) · Med Decis Making · doi 10.1177/0272989X9301300409
- State-transition modeling: a report of the ISPOR-SMDM Modeling Good Research Practices Task Force--3 (opens in a new tab) · Value Health · doi 10.1016/j.jval.2012.06.014
- Cost-Effectiveness of Sotagliflozin in SOLOIST-WHF (opens in a new tab) · JACC Heart Fail · doi 10.1016/j.jchf.2024.04.018
- Cost-Effectiveness of Lecanemab for Individuals With Early-Stage Alzheimer Disease (opens in a new tab) · Neurology · doi 10.1212/WNL.0000000000209218