Lasso
Lasso penalizes the absolute sizes of fitted slopes, so some slopes can become exactly zero while others shrink.
Fit
When to use it, and when not to
Use it when
- You want a prediction model that can discard some candidate predictors.
- You can validate the full selection and tuning process.
Not when
- Every candidate predictor must remain in the model. Consider ridge shrinkage. Read: Uniform shrinkage
- You need causal confounder selection. Choose the adjustment set from the causal question. Read: Directed acyclic graphs
Before you start
What you need
- Outcome, prediction time and intended useoutcomeTypeChoose continuous or binary outcomes before the loss function. These worked examples use squared-error loss.
- Predictor coding, scaling and missing-data planFit scaling and imputation on training data only. Under MAR, assess imputation and include a sensitivity analysis if MNAR is plausible.
- Candidate parameters and validation planPlan the full sample for prediction development. Riley 2019 separates outcome types. Van Smeden shows why events per variable (EPV) is inadequate.
- A selected predictor set need not be stable across samples.How to check: Repeat the full procedure in resamples and inspect changes, especially among closely related predictors.
- Training data cover the intended prediction setting.How to check: Check who was included and which predictors are available at prediction time.
- Performance assessment includes every fitted step.How to check: Fit imputation, scaling and penalty tuning within training folds. Use an outer assessment or bootstrap validation for the full procedure.
Plain language
How it works
For a continuous outcome, lasso adds an absolute-slope penalty to squared prediction errors. This can place slopes exactly at zero. With orthogonal, unit-norm predictors, soft-thresholding subtracts the penalty from each slope magnitude and floors it at zero. Other designs require a joint fit.
Friedman and colleagues give the fitting algorithm; Tay and colleagues extend penalized generalized linear models. Correlated predictors may substitute for each other, making selection unstable. Report the candidate set and the full tuning procedure. Refit ordinary regression after selection only with an inference plan that addresses selection.
Worked example
A worked example
- Setting
- Two centered predictors are orthogonal, each with sum of squares 1. Their outcome cross-products are 2 and 0.5.
- Question
- Which slopes remain under a fixed lasso penalty?
- Approach
- Minimize half the residual sum of squares plus lambda times the sum of absolute slopes. Fix lambda at 1; leave the intercept unpenalized.
- Result
- The first slope changes from 2 to 1, a change of -1. The second changes from 0.5 to 0.
- What it means
- One slope is retained and one is removed. This toy orthogonal fit does not show that the removed predictor lacks clinical value.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
A zero lasso coefficient does not prove that a predictor has no effect. Selection can change across samples, especially with correlated predictors. Ordinary post-selection Wald intervals ignore how predictors and tuning were chosen.
- Treating events per variable as the sample-size method.Instead: Events per variable (EPV) is a minimum floor, not adequate planning. Use outcome-specific prediction criteria or simulation, including all candidate parameters.
- Reporting ordinary unpenalized Wald intervals for penalized slopes.Instead: Coefficient inference needs a justified penalty-aware profile-likelihood or Wald method. Ordinary intervals after tuning or selection ignore that process.
- Selecting confounders by their predictive strength.Instead: Choose causal adjustment from the question and a causal model. Prediction shrinkage does not identify causal effects.
Inside Mallard
How Mallard uses this
Plans do not link to this method yet. No method or sizing entry is mapped to this new article.
analysis.lasso, the id a plan cue records.Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Minimum sample size for developing a multivariable prediction model: Part I - Continuous outcomes (opens in a new tab) · Stat Med · doi 10.1002/sim.7993
- Regularization Paths for Generalized Linear Models via Coordinate Descent (opens in a new tab) · J Stat Softw
- Elastic Net Regularization Paths for All Generalized Linear Models (opens in a new tab) · J Stat Softw · doi 10.18637/jss.v106.i01
- Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes (opens in a new tab) · Stat Med · doi 10.1002/sim.7992
- Sample size for binary logistic prediction models: Beyond events per variable criteria (opens in a new tab) · Stat Methods Med Res · doi 10.1177/0962280218784726