Q&A library Python

How do I select rows where a value falls between two limits in pandas?

Python Beginner betweenfilteringranges

Short answer

Use between, which is clearer than writing two comparisons joined by an ampersand. It includes both endpoints by default, which is worth checking against what you mean.

The short version

Use between(low, high), which returns a boolean column you can filter on. It is clearer than writing two comparisons joined with &. By default it includes both endpoints, so check that matches your intent.

The comparison

import pandas as pd

s = pd.Series([1, 5, 10])

s.between(2, 9).tolist()   # [False, True, False]

The long-hand equivalent is noisier and easier to get wrong:

((s >= 2) & (s <= 9)).tolist()

Both work, but between says what it means, and there are no brackets to misplace.

Filtering a frame

df[df["amount"].between(10, 100)]

Controlling the endpoints

The inclusive argument decides whether the limits themselves count.

s.between(2, 9, inclusive="both")     # default, 2 and 9 both included
s.between(2, 9, inclusive="neither")  # strictly between
s.between(2, 9, inclusive="left")     # includes 2, excludes 9

This matters more than it seems. Bucketing values into ranges with both at every boundary means a value on a boundary lands in two buckets, double-counting it. For contiguous bands, use left so each value belongs to exactly one.

It works on dates too

between compares any orderable type, so it filters date ranges naturally, which is one of its most common uses.

mask = df["order_date"].between("2021-01-01", "2021-03-31")
df[mask]

The strings are converted for comparison, provided the column is a real datetime. If the column is still text, this comparison is alphabetical and silently wrong, which is another reason to parse dates at read time.

And on text

Between works on strings by alphabetical order, which occasionally surprises people who did not intend that.

pd.Series(["apple", "mango"]).between("a", "b")   # [True, False]

Useful for a range of codes, but not something to reach for by accident.

The missing-value behaviour

NaN compares as False in between, so missing values are excluded from the result rather than raising. That is usually right, but be aware they are being silently dropped from your filter, and count them separately if their number matters.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project