Q&A library Python

How do I group dates by month or quarter without losing the year in pandas?

Python Intermediate datetimegroupingperiod

Short answer

Use dt.to_period("M") to get a year-and-month value, so January 2021 and January 2022 stay separate. Grouping by dt.month alone merges the same month across different years.

The short version

Use dt.to_period("M") (or "Q" for quarters), which produces a value carrying both the year and the period, so 2021-01 and 2022-01 remain distinct. Grouping by dt.month alone gives you just the month number, which silently merges the same month from different years into one bucket.

The mistake

import pandas as pd

df = pd.DataFrame({
    "when": pd.to_datetime(["2021-01-15", "2022-01-20"]),
    "amount": [10, 20],
})

df.groupby(df["when"].dt.month)["amount"].sum()

This returns a single row for month 1 with a total of 30, combining January 2021 and January 2022. The year has been thrown away. That is occasionally what you want (a seasonal profile), but it is usually a bug.

The fix

df["period"] = df["when"].dt.to_period("M")
print(df["period"].astype(str).tolist())   # ['2021-01', '2022-01']

df.groupby("period")["amount"].sum()

Now each year's January is its own group, because the period carries both parts.

Other period frequencies

to_period accepts the usual codes: "Q" for quarter, "Y" for year, "W" for week.

df["when"].dt.to_period("Q")   # e.g. 2021Q1

Periods sort chronologically and print readably, which makes them well suited to grouping keys and axis labels.

Period versus timestamp

A Period represents a span of time (the whole of January 2021), whereas a Timestamp is an instant. That distinction is exactly what you want for grouping: every day in January maps to the same period. To convert back to a timestamp (for plotting, say), use dt.to_timestamp(), which gives the start of each period.

df["month_start"] = df["when"].dt.to_period("M").dt.to_timestamp()

The alternative with Grouper or resample

You can achieve the same monthly bucketing with pd.Grouper(key="when", freq="ME") or with resample on a datetime index. These are equivalent for aggregation. to_period is handy when you want the period as an actual column value you can keep, display and sort, rather than only as a grouping mechanism.

When grouping by month number is right

Deliberately merging across years is legitimate when you want a seasonal pattern: "how do sales typically look in January, across all years". In that case dt.month is correct, and dt.month_name() gives readable labels. The point is to make that choice consciously, rather than losing the year by accident.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project