Q&A library Python

How do I summarise values by group in pandas?

Python Beginner aggregationgroupbypandas

Short answer

Use groupby with an aggregation such as sum or mean, then reset_index if you want the grouping column back as an ordinary column. Use agg to produce several named summaries at once.

The short version

Group the rows with groupby, then apply an aggregation like sum, mean or count. The grouping column becomes the index of the result, which surprises people, so call reset_index if you want it back as an ordinary column.

A worked example

import pandas as pd

sales = pd.DataFrame({
    "city": ["Leeds", "Leeds", "Bath", "Bath"],
    "amount": [100, 150, 80, 120],
})

totals = sales.groupby("city")["amount"].sum()
print(totals)

This prints a Series indexed by city:

city
Bath     200
Leeds    250
Name: amount, dtype: int64

The common surprise

Notice that city is now the index, not a column. If you try totals["Leeds"] you get a value, but totals["city"] raises a KeyError. To get an ordinary two column table back, reset the index:

totals = sales.groupby("city")["amount"].sum().reset_index()

Several summaries at once

When you want more than one figure per group, use agg with named outputs. This is clearer than grouping several times and joining the results.

summary = sales.groupby("city").agg(
    total=("amount", "sum"),
    average=("amount", "mean"),
    orders=("amount", "count"),
).reset_index()
print(summary)
    city  total  average  orders
0   Bath    200    100.0       2
1  Leeds    250    125.0       2

Why this is better than a loop

People sometimes loop over unique values and filter the frame once per group. That works but is slow and hard to read, and it repeats the filtering logic. groupby does the split, apply and combine in one optimised pass, so prefer it for anything beyond a handful of rows.

A note on grouping by several columns

Pass a list to group by more than one key. The result then has a MultiIndex, and reset_index turns each level back into its own column.

sales.groupby(["city", "amount"]).size().reset_index(name="count")

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project