Q&A library Python

Why does loc return several rows for a single label in pandas?

Python Advanced duplicatesindexloc

Short answer

A duplicated index label matches every row that carries it, so loc returns a frame rather than a row. Check the index is unique before relying on label lookups.

The short version

If the index has duplicate labels, .loc[label] returns every row carrying that label, which is a DataFrame, not a single row. Code expecting one row then breaks in confusing ways. Check that the index is unique before relying on label lookups.

Seeing it

import pandas as pd

df = pd.DataFrame({"value": [1, 2]}, index=["a", "a"])

len(df.loc["a"])   # 2 , not 1

Both rows are labelled a, so both are returned. With a unique index, .loc["a"] would give a Series representing one row. With duplicates, it gives a DataFrame. The type of the result changes depending on the data, which is what makes this so awkward: code that works on one dataset breaks on another.

Check for uniqueness

df.index.is_unique     # False here
df.index.duplicated().sum()   # how many duplicate labels

Run this before setting an index you intend to look up by. It takes a moment and prevents a whole class of bug.

Where duplicate indexes come from

Most often, concat without ignore_index=True. Each source frame carries its own 0, 1, 2 index, and stacking them produces repeated labels.

pd.concat([a, b])                      # duplicate labels
pd.concat([a, b], ignore_index=True)   # clean 0..n index

They also arise from set_index on a column that is not actually unique, which usually means your assumption about the data was wrong and is worth investigating rather than papering over.

The knock-on effects

A duplicated index does more than confuse .loc. It breaks alignment in arithmetic between Series, can make join multiply rows, and produces ValueError: cannot reindex on an axis with duplicate labels in operations that need a unique index. The error messages appear far from the cause, which is why checking early pays off.

Fixing it

If the labels are meaningless leftovers, discard them.

df = df.reset_index(drop=True)

If the index is meant to be a key and is not unique, the duplicates are a data problem. Look at them before deciding.

df[df.index.duplicated(keep=False)]

They may be genuine repeated records to aggregate, or accidental duplicates to remove.

Avoid meaningful indexes unless they earn it

The deeper lesson is that a meaningful index is a commitment: it must be unique and sorted to behave well. Unless you need label lookups, alignment or time-series tools, keeping the data in ordinary columns and filtering with boolean conditions avoids the entire category of index problems, at no cost.

Enjoying these?

Go from piecing together answers to real fluency.

Our courses take you from looking things up to doing the work with confidence, step by step on real data.

Browse the courses