Q&A library Python

How do I select data from a MultiIndex in pandas?

Python Advanced locmultiindexxs

Short answer

Use loc with a tuple to select across index levels, and xs to slice on an inner level by name. Sort the index first, because selecting from an unsorted MultiIndex is slow and can raise.

The short version

Use .loc with a tuple to select down the levels of a MultiIndex, and .xs to slice on a single inner level by name. Sort the index with sort_index first, because selecting from an unsorted MultiIndex is inefficient and sometimes raises a performance-related error. A MultiIndex is just an index with more than one level, so the usual tools still apply, with a little extra syntax.

Setting up a MultiIndex

import pandas as pd

df = pd.DataFrame({
    "city": ["Leeds", "Leeds", "Bath"],
    "year": [2020, 2021, 2020],
    "value": [1, 2, 3],
}).set_index(["city", "year"]).sort_index()

Selecting on the outer level

Selecting by the first level alone works like an ordinary .loc.

df.loc["Leeds"]        # both Leeds rows, year becomes the index

Selecting a specific combination

Pass a tuple to pin more than one level.

df.loc[("Leeds", 2021)]   # the single Leeds 2021 row

The tuple reads as (outer, inner), so this is the row where city is Leeds and year is 2021.

Slicing on an inner level

To select by the inner level while ignoring the outer, use xs (cross-section), naming the level.

df.xs(2020, level="year")   # every city's 2020 row

This is the neat way to answer "all cities in 2020", which is awkward with .loc alone because the year is not the first level.

Why you must sort the index

An unsorted MultiIndex makes label lookups slow, and partial slicing can raise UnsortedIndexError. Calling sort_index() once after building the frame fixes both. If a selection unexpectedly errors or crawls, an unsorted index is the usual cause.

Selecting with slices across levels

For range-style selection across levels, pd.IndexSlice gives readable syntax.

idx = pd.IndexSlice
df.loc[idx[:, 2020], :]   # all cities, year 2020

The : in the first position means "every value of the outer level", and 2020 pins the inner level. This is the general tool once your selections get more complex than a single tuple.

A word of caution

MultiIndexes are powerful but easy to overuse. If you find the index syntax fighting you at every step, consider reset_index() to turn the levels back into ordinary columns and using plain boolean filtering, which many people find clearer for day-to-day work.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project