Q&A library Python

What is the difference between where, mask and clip in pandas?

Python Advanced clipmaskwhere

Short answer

where keeps values where the condition is True and replaces the rest, mask does the exact opposite, and clip caps values at a lower and upper bound. The inverted logic of where is the usual confusion.

The short version

where keeps the values where the condition is True and replaces everywhere it is False. mask is the mirror image: it replaces where the condition is True. clip is simpler, capping values at a floor and ceiling. The inverted sense of where is what trips people up, because it reads like a filter but behaves like a keep-or-replace.

Seeing all three

import pandas as pd

v = pd.Series([1, 50, 200])

v.where(v < 100, 0)    # [1, 50, 0]   keeps where True, replaces the rest
v.mask(v < 100, 0)     # [0, 0, 200]  replaces where True
v.clip(0, 100)         # [1, 50, 100] caps at the bounds

Read where(cond, other) as "keep the value where the condition holds, otherwise use other". Read mask(cond, other) as "blank out the value where the condition holds".

The confusion with filtering

A boolean filter v[v < 100] removes rows, returning a shorter Series. where does not remove anything: it keeps the same length and substitutes values instead. That difference matters when the result must line up with other columns, which is exactly why where exists.

v[v < 100]           # [1, 50]        shorter, rows dropped
v.where(v < 100)     # [1, 50, NaN]   same length, value replaced

With no other given, where fills with NaN, which is often what you want when marking values as invalid without losing the row.

Why clip is not just a special case

clip caps at both ends in one call, which would otherwise take two where calls.

v.clip(lower=0, upper=100)

Use it for winsorising outliers or enforcing a valid range, for example bounding a percentage to 0 to 100. It is clearer than nested conditionals and states the intent directly.

Choosing between them

Use where when the natural way to say it is "keep these, replace the others". Use mask when it is "replace these". They are interchangeable with an inverted condition, so pick whichever makes the sentence read naturally, since v.where(v < 100, 0) and v.mask(v >= 100, 0) do the same thing.

Replacing with values from another column

The other argument can be a Series, not just a scalar, which allows conditional column merging without a loop.

df["final"] = df["preferred"].where(df["preferred"].notna(), df["fallback"])

This takes the preferred value when present and falls back to another column when not, which is a neat, vectorised alternative to a row-wise apply.

They work on whole frames too

All three apply to a DataFrame as well as a Series, applying the condition element by element across every column, which is handy for bounding or blanking an entire numeric table at once.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project