How do I calculate the time between rows in pandas?
Short answer
Subtract datetime columns or use diff to get a Timedelta, then convert to the unit you want with dt.total_seconds. Sort first and group so gaps do not span across groups.
The short version
Subtracting two datetime values, or calling diff on a datetime column, gives a Timedelta. Convert it to a number in your chosen unit with dt.total_seconds() (then divide) or dt.days. Sort the data by time first, and group when the gaps should not cross between groups.
The gap between consecutive rows
import pandas as pd
df = pd.DataFrame({
"ts": pd.to_datetime([
"2021-01-01 09:00", "2021-01-01 11:30", "2021-01-02 09:00",
])
})
df["gap_hours"] = df["ts"].diff().dt.total_seconds() / 3600
print(df["gap_hours"].tolist()) # [nan, 2.5, 21.5]
diff gives the Timedelta from the previous row, total_seconds() turns it into seconds, and dividing by 3600 gives hours. The first row is NaN because there is no earlier row to measure from.
Why total_seconds and not just the Timedelta
A Timedelta prints nicely but is awkward to compare or plot as a number. Converting to a plain number in a known unit (/ 3600 for hours, / 86400 for days, or dt.days for whole days) gives you something you can filter, average and chart. Choose the unit that suits the scale of your gaps.
Duration between two specific columns
If each row has a start and an end, subtract the columns directly.
df["duration_min"] = (df["end"] - df["start"]).dt.total_seconds() / 60
Both columns must be real datetimes, so convert them with pd.to_datetime first if they arrived as text.
The ordering trap
diff measures against the previous row in current order, so unsorted data gives meaningless gaps. Sort by the timestamp first.
df = df.sort_values("ts")
df["gap"] = df["ts"].diff()
Keeping gaps within groups
Across several groups, a plain diff would measure from the last row of one group to the first of the next. Group so each gap stays inside its own group, and the first row of each group correctly becomes NaN.
df["gap"] = df.sort_values(["user", "ts"]).groupby("user")["ts"].diff()
Building a Timedelta yourself
To add or compare against a fixed duration, construct one with pd.Timedelta.
df["ts"] + pd.Timedelta(hours=1) # shift every time forward an hour
df[df["gap_hours"] > 24] # rows more than a day apart
This is the clean way to express thresholds like "more than a day" without hand-computing seconds.