When should I use agg, transform, filter or apply on a groupby?

The short version

Choose by the shape you want back. agg reduces each group to one row. transform returns a value for every original row. filter keeps or drops whole groups, changing the row count but not the columns. apply can do anything but is slow and its output shape is unpredictable, so use it last.

agg: one row per group

Use it for summaries.

import pandas as pd

df = pd.DataFrame({
    "city": ["Leeds", "Leeds", "Bath"],
    "amount": [1, 2, 3],
})

df.groupby("city").agg(total=("amount", "sum"))

Three rows in, two rows out (one per city). This is what you want for a report.

transform: one value per original row

Use it when the group-level figure must be attached back to every row, for example each row's share of its group total.

df["city_total"] = df.groupby("city")["amount"].transform("sum")

Three rows in, three rows out. The Leeds rows both carry 3, the Bath row carries 3. This is the tool people miss most often, reaching instead for agg and then a merge to glue the result back on.

filter: whole groups in or out

Use it when a property of the group decides whether its rows survive.

df.groupby("city").filter(lambda g: len(g) > 1)

Only the Leeds rows remain, because Bath's group failed the test. The columns are unchanged; only the row count differs.

apply: the fallback

apply hands you the whole group as a DataFrame and accepts whatever you return, so it can do things the others cannot, such as returning several rows per group with new columns. That flexibility costs speed (it runs Python per group) and clarity (the output shape depends on what your function happened to return).

df.groupby("city").apply(lambda g: g.nlargest(1, "amount"), include_groups=False)

Before reaching for it, ask whether agg, transform or filter fits. Usually one does.

The decision in one line

Ask what shape you want. One row per group, use agg. One value per input row, use transform. Fewer rows but whole groups removed, use filter. Anything else, and only then, apply.

The most common mistake

Using agg when you meant transform, then struggling to merge the summary back onto the original rows.

totals = df.groupby("city")["amount"].sum()   # indexed by city
df["city_total"] = totals                      # misaligns, gives NaN

The aggregate is indexed by city, not by the original rows, so assigning it back does not line up. transform avoids the whole problem by returning a result already aligned to the input.

Performance ordering

agg and transform with built-in function names ("sum", "mean") are fastest, because pandas uses optimised implementations. Passing a lambda to either is slower. apply is slowest of all. So prefer a named function string where one exists, which is both quicker and clearer.