How do I handle missing values (NaN) in pandas?

The short version

Missing values show up as NaN. Find them with isna, drop the affected rows with dropna, or replace them with fillna. There is no single right choice: dropping loses data, filling invents it, so pick the one that matches what the missing value actually means.

Finding them

import pandas as pd
import numpy as np

temps = pd.DataFrame({"temp": [20.0, np.nan, 22.0]})

print(temps["temp"].isna().sum())   # 1

isna returns a boolean column, and summing it counts the missing entries because True counts as one.

Dropping

dropna removes rows with any missing value by default. Be careful, because on a wide frame a single missing cell drops the whole row.

temps.dropna()                      # rows with no NaN
temps.dropna(subset=["temp"])       # only consider the temp column

Filling

fillna replaces missing entries with a value you choose. A constant, the column mean and forward fill are the common options.

temps["temp"].fillna(0)                    # constant
temps["temp"].fillna(temps["temp"].mean()) # column mean
temps["temp"].ffill()                      # carry the last value forward

The trap that catches everyone

NaN is not equal to anything, including itself, so comparisons silently fail.

np.nan == np.nan   # False

This means you cannot test for missing values with == np.nan. Always use isna or notna. It also means aggregations skip NaN by default: mean and sum ignore missing values rather than returning NaN, which is usually what you want but worth knowing.

Choosing well

Ask what the gap represents. A missing temperature reading is probably best filled or interpolated. A missing identifier usually means the row is unusable and should be dropped. Filling a numeric column with zero can quietly distort an average, so only do it when zero is the true meaning. Whatever you choose, do it explicitly so the next reader can see the decision.