How do I find rows in one DataFrame that are not in another in pandas?

The short version

To keep the rows of one frame whose key is absent from another, filter with isin and negate it with ~. That handles a single key column cleanly. When the match spans several columns, or you want the merged columns too, use a left merge with indicator=True and keep the left_only rows. This operation is called an anti-join.

The simple case: one key column

import pandas as pd

left = pd.DataFrame({"id": [1, 2, 3], "value": ["a", "b", "c"]})
right = pd.DataFrame({"id": [2]})

missing = left[~left["id"].isin(right["id"])]
print(missing["id"].tolist())   # [1, 3]

left["id"].isin(right["id"]) marks rows whose id appears in right, and the leading ~ flips it to keep the rows that do not. This is the fastest and clearest anti-join for a single key.

The general case: merge with indicator

For a match on several columns, or when you want the result to carry columns from both frames, a left merge with an indicator is the robust approach.

merged = left.merge(right, on="id", how="left", indicator=True)
anti = merged[merged["_merge"] == "left_only"].drop(columns="_merge")
print(anti["id"].tolist())   # [1, 3]

The _merge column records where each row matched, so keeping left_only gives exactly the rows with no partner on the right. Dropping the indicator column tidies the result.

Why not a plain merge and check for NaN

A tempting approach is to merge and then look for NaN in a right-hand column. That breaks when the right-hand data itself contains genuine NaN values, because you cannot tell a real missing value from an unmatched row. The indicator column is unambiguous, which is why it is the safe general method.

Matching on multiple columns

isin compares one column, so for a compound key, either merge as above, or build a combined key first.

key = ["city", "date"]
merged = left.merge(right[key], on=key, how="left", indicator=True)
merged[merged["_merge"] == "left_only"]

Passing only the key columns of right avoids pulling in its other columns when you just want the anti-join.

Watch for duplicate keys

If the right frame has duplicate keys, a merge can multiply the left rows before you filter, changing your counts. Deduplicate the right frame's keys first with drop_duplicates when you only care about membership, so the anti-join returns one row per original left row.

The reverse and the symmetric case

Swap the frames to find rows in the right that are absent from the left. To find rows unique to either side at once, use how="outer" with indicator=True and keep everything that is not both.