What is the difference between first and head in a pandas groupby?
Short answer
first returns the first non-missing value in each column, skipping NaN, while head returns the first row as it is, missing values included. They differ exactly when the first row has gaps.
The short version
groupby(...).first() returns the first non-missing value in each column, skipping over NaN. groupby(...).head(1) returns the first row of each group, exactly as it is, missing values and all. They differ precisely when the first row contains a gap, and that difference silently changes your results.
Seeing the difference
import pandas as pd
import numpy as np
df = pd.DataFrame({
"city": ["Leeds", "Leeds"],
"value": [np.nan, 2.0],
})
df.groupby("city")["value"].first().tolist() # [2.0] , skipped the NaN
df.groupby("city").head(1)["value"].tolist() # [nan] , took the first row as-is
first looked past the missing value and found 2.0. head(1) returned the first row, which happens to be missing.
Which one do you want
Ask what the question really is.
If you want "the first recorded value for each city", first is right, because a missing entry is not a recorded value.
If you want "the first row for each city", head(1) is right, because you want that row, whatever it contains, along with all its other columns.
The distinction matters most when you are building a summary. Using head(1) to get a per-group starting value will hand you NaN whenever the first row happens to be incomplete, quietly putting gaps into your summary.
first operates per column
Another important difference: first works column by column, so the values it returns may come from different rows.
df.groupby("city").first()
If the first row has a missing amount and the second has a missing date, first will take the amount from row two and the date from row one, producing a composite that matches no actual record. That is sometimes exactly what you want (the earliest known value of each field) and sometimes a fiction.
head(1) never does this: it returns a real, whole row.
Sort first, always
Both depend on the current row order, so "first" means nothing unless the rows are sorted the way you intend.
df = df.sort_values(["city", "date"])
Without the sort, you get whichever row happened to be first in the file, which is arbitrary.
The relatives
last mirrors first, taking the final non-missing value, and tail(1) mirrors head(1). The same distinction applies, so choose on the same grounds. nth(0) gives the row at a position and, by default, behaves like head, but it accepts a dropna argument to control the behaviour explicitly, which is worth knowing when you want the choice to be visible in the code.