Q&A library Python

How do I calculate a weighted average in pandas?

Python Advanced groupbystatisticsweighted-average

Short answer

Use numpy.average with a weights argument, or multiply and divide the sums yourself. A plain mean ignores the weights, which quietly gives the wrong answer when the quantities differ.

The short version

Compute a weighted average with numpy.average(values, weights=w), or equivalently as the sum of value times weight divided by the sum of weights. A plain mean treats every row equally, so it gives the wrong answer whenever the rows represent different quantities, for example prices at different volumes.

Why a plain mean is wrong here

Suppose two prices, 10 (one unit) and 20 (three units). The plain mean is 15, but most of the units sold at 20, so the true average price paid is higher. The weighted average accounts for that.

import numpy as np
import pandas as pd

df = pd.DataFrame({"price": [10, 20], "qty": [1, 3]})

weighted = np.average(df["price"], weights=df["qty"])
print(weighted)   # 17.5, not 15

The 20 counts three times as much as the 10, pulling the average up to 17.5.

Doing it without numpy

The same result comes from the definition, which is useful when you want to stay in pure pandas.

(df["price"] * df["qty"]).sum() / df["qty"].sum()   # 17.5

This is worth knowing because it makes the logic explicit: total value divided by total weight.

Weighted average per group

For a weighted average within each group, apply the calculation group by group.

g = pd.DataFrame({
    "city": ["Leeds", "Leeds", "Bath"],
    "price": [10, 20, 5],
    "qty": [1, 3, 2],
})

result = g.groupby("city").apply(
    lambda x: np.average(x["price"], weights=x["qty"]),
    include_groups=False,
)
print(result)

Each group gets its own value times weight sum divided by its own weight sum. The include_groups=False argument keeps recent pandas from warning about the grouping column being passed into the function.

The zero-weight trap

If a group's weights sum to zero, the division is undefined and you get an error or NaN. This happens with all-zero quantities, so guard against it if that is possible in your data, for example by filtering out zero-weight rows first.

When the weighted and plain means agree

They coincide only when every weight is equal. So if your rows genuinely represent equal-sized units, the plain mean is correct and simpler. The moment the rows carry different quantities, sizes or frequencies, reach for the weighted average, because the plain mean silently misrepresents the data.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project