Q&A library Python

How do I make a category column sort in a meaningful order in pandas?

Python Advanced categoricalorderingsorting

Short answer

Convert to an ordered category with the levels listed in the order you want, so sorting and comparison follow your order rather than the alphabet. Note that pulling out a single value returns a plain string again.

The short version

Text sorts alphabetically, so "high" comes before "low", which is rarely what you want for a scale. Convert the column to an ordered category with the levels listed in your intended order. Sorting, comparison and max then follow that order. One catch: extracting a single element gives you back a plain string, which compares alphabetically again.

The problem

import pandas as pd

s = pd.Series(["low", "high", "medium"])
s.sort_values().tolist()   # ['high', 'low', 'medium'] , alphabetical, useless

Alphabetical order tells you nothing about the scale.

Making it an ordered category

List the categories in the order you mean, and mark it ordered.

c = s.astype("category").cat.set_categories(
    ["low", "medium", "high"], ordered=True
)

print(c.sort_values().tolist())   # ['low', 'medium', 'high']
print(c.max())                     # 'high'
print((c > "low").tolist())        # [False, True, True]

Now sorting follows the scale, max returns the genuinely highest level, and comparisons like > "low" work as you would expect.

The gotcha when you pull out one value

Indexing a single element returns the underlying plain string, not an ordered category scalar. So comparing two extracted values reverts to alphabetical comparison.

c[1] > c[0]          # 'high' > 'low' -> False, a plain string comparison!

This is easy to misread as the ordering not working. The order is applied at the Series level (sorting, comparisons against the Series, min, max), not to values you have pulled out individually. Compare on the Series, or compare positions in c.cat.categories, rather than comparing two extracted strings.

Why bother with categories at all

Two reasons. Correct ordering, as above, which matters for any scale (sizes, priorities, bands). And memory: a category stores each distinct label once and keeps small integer codes per row, so a repetitive text column shrinks dramatically. On a large frame, converting such columns is one of the biggest easy memory wins available.

Ordering a groupby or a plot

An ordered category also fixes the order of groups and of plot axes, so your bars or summary rows appear low, medium, high rather than alphabetically. Setting the category order once at the top of a pipeline saves manual reordering at every output.

Sorting by a custom order without categories

If you only need one ad hoc sort, you can map the labels to numbers and sort on that instead.

order = {"low": 0, "medium": 1, "high": 2}
s.iloc[s.map(order).argsort()]

This works, but if the order is intrinsic to the column, making it an ordered category is cleaner, because the meaning travels with the data instead of being re-applied at every sort.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project