How do I choose the decision threshold for a classifier?

The short version

predict applies an arbitrary 0.5 threshold to the predicted probability. That is rarely the right cut-off, especially with imbalanced classes. Use predict_proba to get the probabilities, then choose a threshold that matches the relative cost of a false positive versus a false negative.

The default is a choice you did not make

from sklearn.linear_model import LogisticRegression

model = LogisticRegression(max_iter=500).fit(X_train, y_train)

model.predict(X_test)                     # applies a hidden 0.5 threshold
proba = model.predict_proba(X_test)[:, 1] # the actual probabilities

On an imbalanced test set, thresholding at 0.5 flagged 16 cases as positive, while lowering the threshold to 0.2 flagged 25. Same model, same probabilities, very different behaviour. Nothing about 0.5 is special; it is just the default.

Choosing the threshold deliberately

predictions = (proba >= 0.2).astype(int)

Lower the threshold to catch more positives (higher recall, more false alarms). Raise it to be more selective (higher precision, more misses). The right value depends entirely on which error costs you more.

If missing a fault means a machine breaks, a low threshold is right, and you accept more unnecessary inspections. If every flagged case triggers an expensive investigation, a high threshold is right, and you accept missing some.

Use the precision-recall curve

from sklearn.metrics import precision_recall_curve

precision, recall, thresholds = precision_recall_curve(y_test, proba)

This gives the precision and recall achievable at every possible threshold, so you can see the trade-off explicitly and pick the point that matches your requirements, rather than guessing.

A common approach is to find the lowest threshold that still achieves a required precision, or the threshold that maximises F1 if both errors are roughly equally costly.

Choose on validation data, not on test

The threshold is a parameter, and choosing it by looking at your test performance means the test set has influenced your model. Pick it on a validation set (or within cross-validation), then report the final performance on the untouched test set. Otherwise your reported numbers are optimistic.

Probabilities may not be well calibrated

A predicted probability of 0.8 should mean the event happens about 80 per cent of the time. Some models (particularly tree ensembles and support vector machines) produce scores that rank cases well but are not calibrated as probabilities. If you intend to interpret the numbers as probabilities, or to set a threshold based on an expected cost, check the calibration first and correct it if needed with CalibratedClassifierCV.

For simply ranking cases and cutting at a threshold, calibration matters less, because only the ordering is used.

The wider point

The model gives you a score. Turning that score into a decision is a business judgement about the relative cost of two kinds of mistake, and it does not belong to the model at all. Leaving it at the default 0.5 is not a neutral choice; it is an unexamined one, and on imbalanced data it usually means the model rarely predicts the rare class at all.