How do I compare a value to several rows back in pandas?

The short version

diff(n) and shift(n) accept a number of periods, so you can compare each row with the one n rows earlier. The first n rows produce NaN, because there is nothing that far back to compare with. The data must be complete and evenly spaced for the offset to mean what you think.

Looking further back

import pandas as pd

s = pd.Series([1, 2, 4, 8])

s.diff().tolist()     # [nan, 1.0, 2.0, 4.0] , change from the previous row
s.diff(2).tolist()    # [nan, nan, 3.0, 6.0] , change from two rows back

With diff(2), row 2 compares itself with row 0 (4 minus 1 is 3), and the first two rows have no comparison available.

The classic use: year on year

For monthly data, comparing with twelve rows back gives a year-on-year change, which strips out seasonality.

monthly["yoy_change"] = monthly["amount"].diff(12)
monthly["yoy_growth"] = monthly["amount"].pct_change(periods=12)

Comparing December with November tells you about Christmas. Comparing this December with last December tells you about the business. That is why the twelve-period offset is the standard measure for seasonal data.

The assumption that makes it work

This only works if there is exactly one row per period, with no gaps. "Twelve rows back" equals "twelve months back" only when every month is present. A single missing month shifts everything, so the comparison silently pairs the wrong periods.

Check before you rely on it.

monthly["date"].diff().value_counts()    # should be one consistent interval

If more than one gap size appears, the series has holes, and a positional offset is unsafe.

The safer alternative for gappy data

Rather than counting rows, join the series to itself on an actual date offset, so the comparison is by date rather than by position.

prev = monthly.copy()
prev["date"] = prev["date"] + pd.DateOffset(years=1)
merged = monthly.merge(prev, on="date", suffixes=("", "_last_year"), how="left")

This pairs each month with the same month a year earlier by date, and correctly produces missing values where the earlier month does not exist, rather than quietly pairing the wrong rows.

Resample first if the data is irregular

If your data is event-level rather than one row per period, resample it to a regular frequency first, and only then use a positional offset.

monthly = events.resample("ME", on="date")["amount"].sum()

Just remember that resample fills empty periods with zero for sum, so an absent month becomes a real-looking zero, and a year-on-year change against it will be wildly misleading. Check the counts before trusting the comparison.