Why is my model's feature importance misleading?
Short answer
Tree impurity importance is biased towards high-cardinality features and is computed on training data. Use permutation importance on held-out data, and remember importance is not causation.
The short version
The built-in feature_importances_ on a tree model is computed from training data and is biased towards features with many distinct values. Prefer permutation importance measured on held-out data. And remember that importance tells you what the model used, not what causes the outcome.
The two measures disagree
from sklearn.ensemble import RandomForestClassifier
from sklearn.inspection import permutation_importance
model = RandomForestClassifier(random_state=0).fit(X_train, y_train)
model.feature_importances_ # impurity-based, from training
result = permutation_importance(model, X_test, y_test, n_repeats=5, random_state=0)
result.importances_mean # permutation-based, from held-out data
On a test dataset, the impurity measure gave weights of roughly 0.03, 0.03, 0.38, 0.14, 0.41 across five features, while permutation importance gave 0.01, 0.02, 0.22, 0.02, 0.22. The two broadly agree on which features matter most, but they disagree substantially on the middle ones, and the impurity measure inflates features that the model can split on freely without them actually helping on new data.
Why impurity importance is biased
It counts how much each feature reduced impurity across the trees. A feature with many distinct values (a continuous variable, or worse, an identifier) gives the tree many possible split points, so it wins splits by chance and accumulates importance even when it carries no real signal. A binary feature has one split point and cannot compete on those terms.
The classic demonstration: add a column of random unique numbers to your data and the impurity measure will often rank it above genuinely useful binary features.
Why permutation importance is better
It shuffles one feature at a time and measures how much the model's score on held-out data drops. If a feature genuinely matters, breaking it hurts performance. If it does not, shuffling changes nothing. This measures what the feature actually contributes to prediction, not how often the tree happened to split on it.
The correlated-feature trap
Permutation importance has its own weakness. If two features are highly correlated, shuffling one leaves the model able to recover the information from the other, so both look unimportant even though the information they carry is essential.
The signature is a set of features that all look useless individually but whose removal together destroys the model. Check the correlations before concluding a feature does not matter.
Importance is not causation
This is the most important point, and the most often ignored. Feature importance describes what the model relied on to predict. It does not tell you that changing the feature would change the outcome.
A model predicting equipment failure might lean heavily on "number of maintenance visits", not because visits cause failure but because failing machines get visited. Acting on that as if it were causal (reducing visits to reduce failures) would be absurd, and yet importance rankings are routinely read this way in reports.
What to do
Report permutation importance on held-out data, state the correlation structure, and describe the results as "what the model used" rather than "what drives the outcome". If the question is genuinely causal, importance rankings cannot answer it, and a different design is needed.