How do I keep only the groups that meet a condition in pandas?

The short version

Use groupby(key).filter(func), where func receives each group as a DataFrame and returns a single True or False. Every row of a passing group is kept, and every row of a failing group is dropped. This is filtration on groups, which is quite different from filtering individual rows.

A worked example

import pandas as pd

df = pd.DataFrame({
    "city": ["Leeds", "Leeds", "Bath"],
    "amount": [1, 2, 3],
})

busy = df.groupby("city").filter(lambda g: len(g) > 1)
print(busy)

    city  amount
0  Leeds       1
1  Leeds       2

Leeds has two rows so it survives whole. Bath has one row, so its row is removed even though nothing is wrong with that individual row. The test was about the group, not the row.

Why this is not row filtering

An ordinary boolean filter tests each row on its own. groupby.filter tests a property of the entire group (its size, its total, its mean) and then keeps or drops all its rows together. Questions like "keep only cities with at least ten orders" or "drop groups whose total is zero" are group tests, and they cannot be written as a plain row condition.

Common conditions

df.groupby("city").filter(lambda g: g["amount"].sum() > 100)   # group total
df.groupby("city").filter(lambda g: len(g) >= 10)               # group size
df.groupby("city").filter(lambda g: g["amount"].mean() > 5)     # group average

The function must return a single boolean, not a Series. Returning anything else raises, which is the usual first mistake.

The faster alternative with transform

filter runs a Python function per group, so on many groups it can be slow. The same result often comes from transform, which is vectorised.

sizes = df.groupby("city")["amount"].transform("size")
busy = df[sizes > 1]

transform("size") broadcasts each group's size back onto every row, and then you use an ordinary row filter. This is faster and, once you know the pattern, just as readable. Prefer it for large frames.

Removing small groups before analysis

The most frequent real use is discarding groups with too little data to be meaningful, so that a group of one does not produce a wildly unstable average.

enough = df.groupby("city").filter(lambda g: len(g) >= 5)

Doing this before computing group statistics prevents tiny groups from dominating your results with noise, which is a genuine analytical improvement, not just tidying.

What filter does not do

It does not change the rows or add columns; it only decides which groups survive. If you want a per-group value attached to each row, that is transform. If you want one summary row per group, that is agg. Keeping the three straight (filter, transform, agg) makes groupby far less confusing.