How do I see all the rows involved in a duplicate, not just the extra copies?
Short answer
duplicated marks only the repeats by default, hiding the first occurrence. Pass keep=False to flag every row that participates in a duplicate, which is what you want when reviewing them.
The short version
duplicated() marks only the second and later copies, so the first occurrence is not flagged. That is right for deleting, but wrong for reviewing, because you cannot see what each duplicate is a copy of. Pass keep=False to flag every row involved in a duplicate, including the first.
The two behaviours
import pandas as pd
df = pd.DataFrame({"key": ["a", "a", "b"]})
df["key"].duplicated().tolist() # [False, True, False]
df["key"].duplicated(keep=False).tolist() # [True, True, False]
The default marks only the repeat. With keep=False, both a rows are flagged, because both are part of a duplicated set.
Why the default is what it is
duplicated() is designed to answer "which rows would I delete to leave one of each". That is why the first occurrence is not marked: it is the one you are keeping. drop_duplicates uses exactly this logic.
Use keep=False for inspection
When you are investigating duplicates rather than removing them, you want to see all the copies side by side to work out which is correct.
df[df.duplicated(subset="key", keep=False)].sort_values("key")
Sorting by the key brings each set of duplicates together, so you can compare them row by row. This is the right first move whenever a count comes out higher than expected.
Choosing which columns define a duplicate
By default every column must match. Usually a duplicate is defined by a key, and the other columns may differ (which is exactly what makes them worth inspecting).
df.duplicated(subset=["customer", "date"], keep=False)
This finds rows that repeat a key even when their other values disagree, which is the case most worth investigating, because it means two records claim to describe the same thing differently.
Counting the problem first
Before doing anything, measure it.
df.duplicated(subset="key").sum() # how many rows would be removed
df.duplicated(subset="key", keep=False).sum() # how many rows are involved
The two numbers differ, and the difference tells you how many distinct keys are duplicated. Both are worth knowing.
Why this matters more than it seems
Duplicate keys are the single most common cause of a merge silently multiplying rows and inflating every total downstream. Finding them before a join, rather than after a wrong number appears in a report, is one of the highest-value checks in data work. Making duplicated(keep=False) part of your routine inspection catches them early.
After inspection
Once you understand the duplicates, drop_duplicates removes them, and sorting first lets you control which copy survives (for example keeping the most recent). Inspect first, then delete, never the other way round.