How do I convert labels to numeric codes in pandas?

The short version

pd.factorize turns a column of labels into integer codes and returns the distinct values alongside, so you can map back. It is useful for compact storage and for joining, but the codes are labels, not quantities, so do not feed them into a model as if they were numbers unless the categories are genuinely ordered.

Factorising

import pandas as pd

s = pd.Series(["a", "b", "a"])
codes, uniques = pd.factorize(s)

print(codes.tolist())   # [0, 1, 0]
print(list(uniques))    # ['a', 'b']

The codes are positions into uniques, so uniques[codes] reconstructs the original values exactly.

The mistake that matters

The codes look numeric, so it is tempting to hand them to a model.

df["city_code"] = pd.factorize(df["city"])[0]   # 0, 1, 2, ...

A linear model now believes city 2 is twice city 1, and that city 1 sits between city 0 and city 2. That is nonsense for unordered categories, and it will quietly distort your results. Codes are identifiers, not measurements.

For an unordered category, one-hot encode instead, giving each category its own indicator column.

pd.get_dummies(df["city"])

When codes are legitimate

Codes are fine when the order is real: small, medium, large, or a rating from one to five. There the numeric spacing carries meaning, and treating the code as a number is defensible (though even then, equal spacing is an assumption worth stating).

They are also fine as a compact internal key: for joining, grouping, or saving memory, where the code is never treated as a quantity.

The related category dtype

astype("category") does something similar but keeps the labels attached, storing codes internally while still behaving like text.

df["city"] = df["city"].astype("category")
df["city"].cat.codes      # the underlying integers, if you need them

This is usually the better choice for a column you will keep working with, because you get the memory saving without losing the labels and without the risk of a bare integer column being mistaken for a measurement.

Missing values

factorize gives -1 to missing values by default, which is easy to overlook and will be treated as a real category if you are not careful. Check for it, or pass use_na_sentinel=False if you want them handled differently.

(codes == -1).sum()   # how many were missing

Mapping back

Keep uniques if you will need to translate the codes back into labels for a report, because the codes alone are meaningless to a reader.

pd.Series(uniques[codes])   # the original labels again