Q&A library Python

How do I create every combination of rows from two DataFrames in pandas?

Python Intermediate cartesiancross-joinmerge

Short answer

Use merge with how="cross" to pair every row of one frame with every row of the other. The result grows as the product of the two sizes, so it can become very large quickly.

The short version

Use merge(other, how="cross") to pair every row of one frame with every row of the other, producing the Cartesian product. There is no key; every left row is combined with every right row. Be careful, because the result has as many rows as the two frames multiplied together, so it grows fast.

A worked example

import pandas as pd

colours = pd.DataFrame({"colour": ["red", "blue"]})
sizes = pd.DataFrame({"size": ["S", "L"]})

print(colours.merge(sizes, how="cross"))
  colour size
0    red    S
1    red    L
2   blue    S
3   blue    L

Two colours times two sizes gives four combinations, every pairing of the two lists. This is the standard way to build a grid of all options.

Why you might want this

A cross join is useful for generating a complete set of combinations to fill in, such as every product and every month, so that periods with no data still appear as rows (with zero or NaN) after you join the real figures on. It is the tool for "make sure every combination exists, even the empty ones".

The size warning

The output size is the product of the inputs, so 1,000 rows crossed with 1,000 rows is a million rows, and crossing three such frames is a billion. It is very easy to exhaust memory by cross joining larger frames than you intended. Always sanity-check the expected size (left rows times right rows) before running a cross join on real data.

The older manual trick

Before how="cross" existed (pandas 1.2), people faked it by adding a constant key column to both frames and merging on it.

colours["_k"] = 1
sizes["_k"] = 1
colours.merge(sizes, on="_k").drop(columns="_k")

This still works and you will see it in older code, but how="cross" is cleaner and expresses the intent directly, so prefer it.

Filling a complete grid

The common real use is building a full calendar of combinations, then left-joining actual data onto it.

grid = products.merge(months, how="cross")
complete = grid.merge(actuals, on=["product", "month"], how="left")

Now every product and month appears, and combinations with no actual data show as NaN, which you can then fill with zero if that is the correct meaning. This guarantees no period is silently missing from a report.

Enjoying these?

Go from piecing together answers to real fluency.

Our courses take you from looking things up to doing the work with confidence, step by step on real data.

Browse the courses