Q&A library Python

How do I aggregate several columns with different functions in pandas?

Python Intermediate agggroupbynamed-aggregation

Short answer

Pass named aggregations to agg, giving each output column a name and the column-function pair that fills it. This produces flat, clearly named columns instead of a two-level header.

The short version

Use named aggregation: pass agg a set of output_name=(column, function) pairs. Each becomes one clearly named column in the result, avoiding the awkward two-level header you get from the dictionary form. It is the clearest way to summarise several columns with different functions in one pass.

Named aggregation

import pandas as pd

orders = pd.DataFrame({
    "city": ["Leeds", "Leeds", "Bath"],
    "amount": [5, 9, 7],
    "items": [1, 3, 2],
})

summary = orders.groupby("city").agg(
    total=("amount", "sum"),
    average=("amount", "mean"),
    orders=("amount", "size"),
    items_sold=("items", "sum"),
).reset_index()
print(summary)
    city  total  average  orders  items_sold
0   Bath      7      7.0       1           2
1  Leeds     14      7.0       2           4

Each output column is named by you, so the result needs no further tidying, and the code documents itself.

The older dictionary form

You will also see this style, which still works.

orders.groupby("city").agg({"amount": ["sum", "mean"], "items": "sum"})

It is more compact, but the result has a two-level column header (('amount', 'sum')), which is awkward to reference and usually needs flattening afterwards. Prefer named aggregation for new code, and keep the dictionary form for quick exploration.

Custom functions

Any function that reduces a group to a single value can be used, including a lambda or a named function.

orders.groupby("city").agg(
    spread=("amount", lambda s: s.max() - s.min()),
)

Named functions are worth preferring when the logic is more than trivial, because a name explains the intent where a lambda does not.

The difference between size and count

size counts every row in the group. count counts only non-missing values in the named column. They differ whenever there are NaN values, and choosing the wrong one silently under- or over-counts.

orders.groupby("city").agg(
    rows=("amount", "size"),      # every row
    with_value=("amount", "count"),  # rows where amount is not NaN
)

Aggregating without grouping

agg also works on a whole frame with no groupby, producing one summary row, which is handy for an overall total alongside your grouped one.

orders.agg(total=("amount", "sum"))

Keep the grouping key as a column

groupby puts the key in the index, so call reset_index() if you want it back as an ordinary column, which is almost always what you want before merging or exporting the summary.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project