Q&A library Python

How do I count distinct values in pandas?

Python Beginner distinctnuniqueunique

Short answer

Use nunique for the number of distinct values and unique to list them. nunique ignores missing values by default, and works per group when combined with groupby.

The short version

nunique() gives the count of distinct values, and unique() returns the distinct values themselves as an array. nunique ignores missing values by default. Combine either with groupby to count distinct values within each group, which is a very common reporting need.

Counting and listing

import pandas as pd

s = pd.Series(["x", "y", "x"])

print(s.nunique())            # 2
print(s.unique().tolist())    # ['x', 'y']

nunique answers "how many different ones", unique answers "which ones", and value_counts answers "how many of each".

Missing values are excluded by default

nunique does not count NaN as a distinct value unless you ask it to.

s.nunique(dropna=False)   # counts NaN as one of the distinct values

This matters when a missing value is itself meaningful in your data, so choose the behaviour deliberately rather than accepting the default without thinking.

Distinct values per group

The common real question is "how many distinct products did each city order", which is a groupby with nunique.

df = pd.DataFrame({
    "city": ["Leeds", "Leeds", "Leeds", "Bath"],
    "product": ["pen", "pen", "mug", "cup"],
})

df.groupby("city")["product"].nunique()
city
Bath     1
Leeds    2

Leeds ordered three rows but only two distinct products, which is exactly the distinction nunique captures and a plain count would miss.

nunique versus count versus size

These three are easy to confuse. size counts every row. count counts rows with a non-missing value. nunique counts distinct non-missing values. On the Leeds group above, size is 3, count is 3, and nunique is 2.

Counting distinct rows, not values

To count distinct combinations across several columns, use drop_duplicates and take the length, or value_counts on multiple columns.

len(df.drop_duplicates(subset=["city", "product"]))   # distinct pairs

Distinct across the whole frame

Calling nunique() on a DataFrame gives the count per column, which is a quick and useful profile of your data: a column with one distinct value is constant and probably useless, and a column where the distinct count equals the row count is likely an identifier.

df.nunique()

This one line is a good first thing to run on an unfamiliar dataset.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project