Ask Mallard
Analysis method

Ridge regression

4 min read

Ridge regression shrinks fitted slopes toward zero by penalizing their squared sizes, often stabilizing predictions when predictors overlap.

Fit

When to use it, and when not to

Use it when

  • You want prediction and need to keep the candidate predictors.
  • Predictors are strongly related and unpenalized slopes are unstable.

Not when

  • You require a sparse model that sets slopes to zero. Consider a selection penalty. Read: Uniform shrinkage
  • Your goal is a causal effect with a defined adjustment set. Use an analysis matched to that estimand. Read: The estimand

Before you start

What you need

Decide these first
  1. Outcome, prediction time and intended useoutcomeTypeChoose continuous or binary outcomes before the loss function. These worked examples use squared-error loss.
  2. Predictor coding, scaling and missing-data planFit scaling and imputation on training data only. Under MAR, assess imputation and include a sensitivity analysis if MNAR is plausible.
  3. Candidate parameters and validation planPlan the full sample for prediction development. Riley 2019 separates outcome types. Van Smeden shows why events per variable (EPV) is inadequate.
It rests on these
  1. The scaling and penalty convention are stated.How to check: The same numeric penalty can mean different shrinkage when software divides the loss by sample size.
  2. Training data cover the intended prediction setting.How to check: Check who was included and which predictors are available at prediction time.
  3. Performance assessment includes every fitted step.How to check: Fit imputation, scaling and penalty tuning within training folds. Use an outer assessment or bootstrap validation for the full procedure.

Plain language

How it works

For a continuous outcome, ridge minimizes squared prediction errors plus a penalty on squared slopes. The intercept is usually left unpenalized. Scale predictors first so units do not decide the penalty. Choose the strength using training data, and assess the full procedure on data kept outside that tuning.

What the fit means

Friedman and colleagues describe ridge within penalized model fitting. Positive ridge penalties generally shrink slopes without selecting a subset. A slope can still be zero because of the data. Riley gives outcome-specific prediction-development criteria; van Smeden shows that an EPV rule cannot establish adequate binary prediction performance.

The method as a flow
A single-method workflow. Fit scaling within training data. Tune squared-slope penalty in training data. Assess the full ridge procedure outside tuning. The diagram describes the analysis steps, not results from any study.Ridge regressionFit scaling within training dataTune squared-slope penalty in training dataAssess the full ridge procedure outside tuningModel assumptions and the study design still determine what the estimate can support.
Conceptual diagram. It shows the structure of the idea, not data from any study.

Worked example

A worked example

Invented for illustration
Setting
A centered continuous outcome is modeled with one centered predictor. The predictor has sum of squares 1 and cross-product with the outcome 2.
Question
How does a fixed ridge penalty change the slope?
Approach
Minimize half the residual sum of squares plus lambda times half the squared slope. Fix lambda at 1; leave the intercept unpenalized.
Result
The unpenalized slope is 2. Ridge gives 1, a change of -1. At centered predictor value 2, the fitted centered outcome is 2.
What it means
The slope is smaller but remains in the model. These chosen inputs illustrate shrinkage; they do not show better prediction in new people.

In practice

Recent studies PubMed search · not vetted

Searching PubMed…

Watch out

Common mistakes

The one to remember

Shrinking coefficients can reduce prediction error, but cannot repair confounding, leakage or a small development sample. Ordinary unpenalized Wald intervals do not become valid ridge intervals merely by using the shrunken coefficients.

  • Treating events per variable as the sample-size method.Instead: Events per variable (EPV) is a minimum floor, not adequate planning. Use outcome-specific prediction criteria or simulation, including all candidate parameters.
  • Reporting ordinary unpenalized Wald intervals for penalized slopes.Instead: Coefficient inference needs a justified penalty-aware profile-likelihood or Wald method. Ordinary intervals after tuning or selection ignore that process.
  • Selecting confounders by their predictive strength.Instead: Choose causal adjustment from the question and a causal model. Prediction shrinkage does not identify causal effects.

Inside Mallard

How Mallard uses this

Plans do not link to this method yet. No method or sizing entry is mapped to this new article.

No Mallard code example points to this page.
Atlas node analysis.ridge_regression, the id a plan cue records.

Related Learn pages

Related designs and methods

Evidence

Sources & further reading

  1. Minimum sample size for developing a multivariable prediction model: Part I - Continuous outcomes (opens in a new tab) · Stat Med · doi 10.1002/sim.7993
  2. Regularization Paths for Generalized Linear Models via Coordinate Descent (opens in a new tab) · J Stat Softw
  3. Elastic Net Regularization Paths for All Generalized Linear Models (opens in a new tab) · J Stat Softw · doi 10.18637/jss.v106.i01
  4. Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes (opens in a new tab) · Stat Med · doi 10.1002/sim.7992
  5. Sample size for binary logistic prediction models: Beyond events per variable criteria (opens in a new tab) · Stat Methods Med Res · doi 10.1177/0962280218784726
Prepared with AI assistance. Source titles, authors, years and available DOIs were checked against PubMed on 2026-11-06. No independent professional approval is claimed. Friedman supports the squared penalty and fitting; Tay supports the penalized GLM framework. Riley covers prediction-development sizing by outcome type; The PubMed and PMC record for Friedman supplies no DOI. van Smeden covers the limits of binary EPV rules.
Last source review: 2026-11-06
AI-generated educational content
Mallard uses reputable methodological and reporting sources, but this material may contain errors or omit important context. Verify important methodological decisions with the cited sources and, when appropriate, a qualified research methodologist or statistician.