Elastic net
Elastic net uses one penalty mixing absolute and squared slopes, allowing both shrinkage and some zero coefficients.
Fit
When to use it, and when not to
Use it when
- You want a sparse prediction model with groups of related predictors.
- You can tune penalty strength and mixing weight inside the training process.
Not when
- You require an interpretable causal effect rather than a prediction rule. Choose an estimand-led analysis. Read: The estimand
- The data cannot support model development and honest validation. Simplify the question or obtain more data. Read: Why events per variable is a floor, not a sample-size target
Before you start
What you need
- Outcome, prediction time and intended useoutcomeTypeChoose continuous or binary outcomes before the loss function. These worked examples use squared-error loss.
- Predictor coding, scaling and missing-data planFit scaling and imputation on training data only. Under MAR, assess imputation and include a sensitivity analysis if MNAR is plausible.
- Candidate parameters and validation planPlan the full sample for prediction development. Riley 2019 separates outcome types. Van Smeden shows why events per variable (EPV) is inadequate.
- Alpha controls the mix and lambda controls overall penalty strength.How to check: State both values and the software convention. Fix the intercept outside the penalty.
- Training data cover the intended prediction setting.How to check: Check who was included and which predictors are available at prediction time.
- Performance assessment includes every fitted step.How to check: Fit imputation, scaling and penalty tuning within training folds. Use an outer assessment or bootstrap validation for the full procedure.
Plain language
How it works
Elastic net is one penalty with two controls. Lambda sets overall strength; alpha sets the share of the absolute-value term. Under the stated convention, alpha zero and one give its limiting forms. Intermediate values can retain groups of correlated predictors while removing others. Group retention is not assured.
Friedman and colleagues describe coordinate-descent fitting; Tay and colleagues extend it across generalized linear models. Scale predictors inside training folds. Tune both controls within training data and assess performance outside that tuning. Report the complete procedure rather than just the final nonzero slopes.
Worked example
A worked example
- Setting
- A centered predictor has sum of squares 1 and outcome cross-product 2.
- Question
- How does a fixed mixed penalty change its slope?
- Approach
- Use half the residual sum of squares plus lambda times alpha times absolute slopes plus lambda times half of one-minus-alpha times squared slopes.
- Result
- With lambda 2 and alpha 0.5, the slope changes from 2 to 0.5, a change of -1.5.
- What it means
- Soft-thresholding subtracts 1 from 2; the squared penalty then divides by 2. This is a chosen penalty, not a validated tuning result.
In practice
Recent studies PubMed search · not vetted
Watch out
Common mistakes
The one to remember
Tuning two penalty controls on the same data used to report performance gives an optimistic score. Keeping correlated predictors together is a tendency, not a guarantee, and a sparse model is not a causal adjustment set.
- Treating events per variable as the sample-size method.Instead: Events per variable (EPV) is a minimum floor, not adequate planning. Use outcome-specific prediction criteria or simulation, including all candidate parameters.
- Reporting ordinary unpenalized Wald intervals for penalized slopes.Instead: Coefficient inference needs a justified penalty-aware profile-likelihood or Wald method. Ordinary intervals after tuning or selection ignore that process.
- Selecting confounders by their predictive strength.Instead: Choose causal adjustment from the question and a causal model. Prediction shrinkage does not identify causal effects.
Inside Mallard
How Mallard uses this
Plans do not link to this method yet. No method or sizing entry is mapped to this new article.
analysis.elastic_net, the id a plan cue records.Related Learn pages
Related designs and methods
Evidence
Sources & further reading
- Minimum sample size for developing a multivariable prediction model: Part I - Continuous outcomes (opens in a new tab) · Stat Med · doi 10.1002/sim.7993
- Regularization Paths for Generalized Linear Models via Coordinate Descent (opens in a new tab) · J Stat Softw
- Elastic Net Regularization Paths for All Generalized Linear Models (opens in a new tab) · J Stat Softw · doi 10.18637/jss.v106.i01
- Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes (opens in a new tab) · Stat Med · doi 10.1002/sim.7992
- Sample size for binary logistic prediction models: Beyond events per variable criteria (opens in a new tab) · Stat Methods Med Res · doi 10.1177/0962280218784726