Q&A library Python

How do I check whether any or all rows meet a condition in pandas?

Python Intermediate allanyvalidation

Short answer

Call any or all on a boolean condition to reduce it to a single True or False. Use them for data validation checks, and remember they operate per column on a DataFrame unless you reduce twice.

The short version

Build a boolean condition, then call .any() to ask "is it true anywhere" or .all() to ask "is it true everywhere". Both reduce a column of booleans to a single answer, which makes them the natural tools for data validation checks before you trust a dataset.

The basic checks

import pandas as pd

df = pd.DataFrame({"amount": [5, 9, -2]})

(df["amount"] < 0).any()    # True, at least one negative
(df["amount"] > 0).all()    # False, not every value is positive

The inner expression produces a boolean column, and any/all collapse it to one value you can act on.

Using them as guards

This is the idiomatic way to assert something about your data before proceeding.

if (df["amount"] < 0).any():
    raise ValueError("negative amounts found")

Catching a violated assumption here, at the point it becomes untrue, is far cheaper than tracing a strange total back through five steps later.

Checking for missing values

A very common pair of checks uses isna with any.

df.isna().any()          # per column: does this column have any NaN
df.isna().any().any()    # a single True or False for the whole frame

Note the double any. On a DataFrame, any() reduces each column separately and returns a Series with one answer per column. A second any() collapses that to one overall answer. Forgetting the second call is a frequent confusion, because you get a Series where you expected a boolean.

The empty-frame trap

On an empty column, all() returns True and any() returns False, which follows the mathematical convention but can surprise you. If a filter produced no rows, an all() check will pass vacuously, so check len(df) too when emptiness would itself be a problem.

The truth-value error

You cannot put a whole column into an if directly.

if df["amount"] > 0:   # ValueError: truth value of a Series is ambiguous

Python does not know whether you mean any, all, or something else, so it refuses. Being explicit with .any() or .all() resolves the ambiguity, which is exactly why the error exists.

Counting rather than testing

If you want to know how many rows violate a condition rather than just whether any do, sum the boolean column, because True counts as one.

(df["amount"] < 0).sum()   # how many negatives

Together, any, all and sum on a condition give you a quick, readable vocabulary for validating data, and they run at full vectorised speed with no loop in sight.

Enjoying these?

Go from piecing together answers to real fluency.

Our courses take you from looking things up to doing the work with confidence, step by step on real data.

Browse the courses