How do I standardise a column to z-scores in pandas?

The short version

Standardise a column by subtracting its mean and dividing by its standard deviation, giving z-scores that express each value as a number of standard deviations from the mean. This puts different columns on a common scale, which many models and comparisons need. Two things to decide: whether to standardise within groups, and that pandas uses the sample standard deviation by default.

The calculation

import pandas as pd

s = pd.Series([10.0, 20.0, 30.0])

z = (s - s.mean()) / s.std()
print(z.tolist())   # [-1.0, 0.0, 1.0]

The middle value sits at the mean, so its z-score is 0. The others are one sample standard deviation below and above.

Why standardise at all

When columns have very different scales (say temperature in tens and sales in thousands), a model or distance calculation can be dominated by whichever column has the larger numbers. Standardising removes the scale, so each column contributes on equal terms. It also makes "how unusual is this value" comparable across columns, because a z-score of 2 means the same thing everywhere.

The sample versus population catch

Pandas std divides by n minus 1 (the sample standard deviation) by default, while some tools and textbooks use n (the population version). If your z-scores differ slightly from another library's, this is usually why. Match the convention you need with the ddof argument.

s.std(ddof=0)   # population standard deviation, divides by n

For standardising a whole dataset before modelling, the difference is tiny on large samples but can matter on small ones, so choose deliberately.

Standardising within groups

Often each group should be standardised against its own mean and spread, not the overall ones. Use groupby with transform.

df["z"] = df.groupby("city")["value"].transform(lambda x: (x - x.mean()) / x.std())

This gives each city's values as z-scores relative to that city, which is the right choice when groups differ systematically and you want to compare positions within each.

The constant-column trap

If a column (or a group) has no variation, its standard deviation is zero and the division produces NaN or infinity. Check for zero-variance columns before standardising, because a single constant column can quietly fill your result with invalid values.

Min-max as an alternative

Z-scores are one way to rescale. If you instead need values squeezed into a fixed range like 0 to 1, use min-max scaling, (x - x.min()) / (x.max() - x.min()). Pick z-scores when you care about distance from the mean in standard deviations, and min-max when you need a bounded range.