How do I fill a column from a second column where the first is missing in pandas?
Short answer
Use fillna with another Series, or combine_first, to take the first column's value where present and fall back to the second where missing. This is the pandas equivalent of SQL COALESCE.
The short version
Pass a Series to fillna, or use combine_first, to take each value from the preferred column and fall back to a second column where the first is missing. This is the pandas equivalent of SQL's COALESCE, and it is vectorised, so there is no need for a row-wise apply.
Filling from another column
import pandas as pd
import numpy as np
df = pd.DataFrame({
"preferred": [1.0, np.nan, 3.0],
"fallback": [10.0, 20.0, 30.0],
})
df["final"] = df["preferred"].fillna(df["fallback"])
print(df["final"].tolist()) # [1.0, 20.0, 3.0]
Where preferred has a value it is kept; where it is missing, the matching value from fallback is used. Alignment is by index, so both columns line up row for row.
combine_first does the same for frames
combine_first applies the same idea across whole DataFrames, filling gaps in one from another.
primary.combine_first(secondary)
Every missing value in primary is filled from the same position in secondary, matching on index and column names. This is useful for merging a partial update over a base table.
Chaining several fallbacks
fillna chains, so you can express a priority order of several columns.
df["final"] = df["first"].fillna(df["second"]).fillna(df["third"]).fillna(0)
Each step fills what remains, ending with a constant default. This reads as a clear priority list and stays vectorised throughout.
The wrong way
The tempting approach is a row-wise apply with an if.
# slow and unnecessary
df["final"] = df.apply(
lambda r: r["preferred"] if pd.notna(r["preferred"]) else r["fallback"],
axis=1,
)
This gives the right answer but runs a Python function per row, so it is hundreds of times slower on a large frame, and it is longer to read than fillna.
Watch the dtypes
If the two columns have different types, the result takes a type that can hold both, which may widen integers to floats. That is the same NaN-forces-float behaviour as elsewhere in pandas. If you need to end up with integers, fill first and then cast, or use the nullable Int64 dtype so integers survive the presence of missing values.
Where this comes up
This pattern is everywhere in real data: a preferred contact method with a backup, a corrected value overriding an original, a joined lookup that only covers some rows. Recognising it as "coalesce" rather than reaching for a loop keeps the code short, fast and obvious to anyone who has written SQL.
Enjoying these?
Go from piecing together answers to real fluency.
Our courses take you from looking things up to doing the work with confidence, step by step on real data.
Browse the courses