How do I find the differences between two DataFrames?

The short version

df.equals(other) returns a single True or False, including a dtype check. df.compare(other) shows exactly which cells differ, side by side. When the two frames have different rows rather than different values, merge with indicator=True instead, to see what is present on each side.

The quick check

import pandas as pd

a = pd.DataFrame({"x": [1, 2]})
b = pd.DataFrame({"x": [1, 2]})

a.equals(b)   # True

Note that equals also compares dtypes, so an integer column and an identical-looking float column are not equal. That strictness is usually helpful: it tells you a type has drifted, which often means a missing value crept in somewhere.

Seeing what differs

a = pd.DataFrame({"x": [1, 2], "y": [3, 4]})
b = pd.DataFrame({"x": [1, 9], "y": [3, 4]})

a.compare(b)

     x
  self other
1  2.0   9.0

compare shows only the cells that differ, with the value from each side. Rows and columns that match are omitted entirely, so on a large frame with a few changes you see exactly the changes and nothing else.

The alignment requirement

compare requires the two frames to have the same shape and labels. If they do not, it raises. That is a deliberate limitation: it compares values, and it cannot compare values when it does not know which rows correspond.

When the rows differ

For frames with different rows, the question is not "which cells differ" but "which rows are on each side". Merge with an indicator.

merged = a.merge(b, on="id", how="outer", indicator=True)

merged[merged["_merge"] == "left_only"]    # in a but not b
merged[merged["_merge"] == "right_only"]   # in b but not a
merged[merged["_merge"] == "both"]         # in both

This is the tool for reconciling two extracts, or for finding out why a refreshed dataset has a different row count from yesterday's.

Comparing floats

Never compare computed floats exactly, because tiny representation errors make identical-looking values unequal. Use a tolerance.

import numpy as np
np.isclose(a["x"], b["x"]).all()

Or, in a test, pandas.testing.assert_frame_equal(a, b, check_exact=False), which applies a sensible tolerance for you.

A practical reconciliation routine

When two versions of a dataset disagree, work through it in order: compare the shapes, then the columns, then use the merge indicator to find rows present on only one side, and only then use compare on the common rows to find differing values. Answering the questions in that order isolates the cause quickly, whereas jumping straight to cell comparison on mismatched frames just produces a confusing error.