Permutation importance with correlated features: what a small score means

What permutation importance measures

Permutation importance measures how a fitted model's score changes when one feature column is randomly shuffled across observations. The model is not retrained. For a score where higher is better, importance is the original score minus the average shuffled score. A larger drop indicates greater reliance under this particular perturbation. The result belongs to a model, dataset and metric; it is not an intrinsic property of the feature. [1]

A practical measurement procedure

First establish that the model predicts adequately on held-out data. Fix the scoring function, then compute the original score and repeatedly permute each feature. Report mean importance and the spread across permutations, together with the number of repeats and random seed. The spread describes shuffle variability for this fitted model and dataset, not all uncertainty from collecting data or retraining. [1]

If the analysis informs feature selection, use validation data and preserve a separate final test set. Record whether the permutation acts on raw columns through a complete pipeline or on transformed columns.

Why correlation changes the interpretation

Correlated predictors may contain overlapping information. Shuffling one can produce a small score drop because the fitted model still uses another predictor carrying similar information. This depends on the model; correlation does not guarantee a small importance. Permutation can also create combinations absent from the real data. [1][2]

The scikit-learn example explores clustering correlated features, keeping a representative and retraining. That answers a different question from inspecting the original model: how a newly trained model behaves with a reduced feature set. [2]

Illustrative example

Suppose a machine-failure model uses two nearly duplicate temperature sensors. Small individual importance values do not establish that temperature is irrelevant. As an additional diagnostic, shuffle both sensor columns using the same row permutation. Their internal pairing remains intact, while their joint relation to the other rows is broken. Comparing this group perturbation with individual perturbations can expose shared reliance. This is a proposed diagnostic, not a measured result; grouping still changes the interpretation.

Limits and responsible conclusions

State the perturbation, feature groups and metric alongside any ranking. A near-zero or negative importance can reflect sampling variability, redundant information or a model that uses a feature poorly. It does not automatically justify removing the variable. Validate removal by retraining and comparing on independent evaluation data. Neither individual nor grouped permutation establishes a causal effect of changing a real-world variable. This article is a source-based synthesis; no local experiment was performed.

Sources

[1] scikit-learn User Guide: Permutation Feature Importance

[2] scikit-learn Example: Permutation Importance with Multicollinear or Correlated Features