Q&A library Python

How do I group by a time period on a date column in pandas?

Python Advanced groupbygroupertime-series

Short answer

Use pd.Grouper with a key and a frequency to group a date column into periods without making it the index. This combines the convenience of resample with grouping by other columns at the same time.

The short version

Use pd.Grouper(key="date", freq="ME") inside groupby to bucket a date column into periods (months, weeks, quarters) while it stays an ordinary column. This is like resample, but it does not require the dates to be in the index, and it lets you group by other columns in the same call.

A worked example

import pandas as pd

df = pd.DataFrame({
    "date": pd.to_datetime(["2021-01-05", "2021-01-20", "2021-02-03"]),
    "amount": [1, 2, 3],
})

monthly = df.groupby(pd.Grouper(key="date", freq="ME"))["amount"].sum()
print(monthly.tolist())   # [3, 3]

The two January rows are summed to 3, and February gives 3. key names the date column, and freq sets the bucket size, using the same aliases as resample (ME for month end, W for week, QE for quarter, and so on).

Why not just resample

resample needs the dates in the index, so you have to set and later reset the index around it. Grouper works directly on a column, which keeps your frame flat and fits naturally into a normal groupby. If your dates are already in the index, resample is slightly shorter; if they are a column, Grouper avoids the index juggling.

The real advantage: grouping by time and category together

The strongest reason to use Grouper is combining a time bucket with other grouping keys in one step.

df.groupby(["city", pd.Grouper(key="date", freq="ME")])["amount"].sum()

This gives the monthly total per city, which resample alone cannot do without extra grouping. Mixing a Grouper with ordinary column keys in the list is the idiomatic way to summarise by category and period at once.

The datetime requirement

The keyed column must be a real datetime, so convert it first with pd.to_datetime. If you pass a text column, Grouper cannot bucket it and raises. This is the same requirement as resample, just checked on a column rather than the index.

Matching the frequency alias to your version

The frequency codes were renamed in pandas 2.2, so month is now ME or MS rather than the deprecated M. If you see a FutureWarning about the alias, switch to the explicit end or start form. On older pandas the short codes still work, so use whichever your version accepts.

Empty periods

Like resample, Grouper includes periods that have no rows, filling a sum with zero. Decide whether an empty month should read as zero or be dropped, because "no data" and "zero" are different claims about that period.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project