Q&A library Python

How do I create a column that depends on another column I just created in pandas?

Python Intermediate assignchaininglambda

Short answer

Inside a single assign call, use a lambda for the second column so it sees the frame including the first. Referring to the original frame directly fails, because that column does not exist yet.

The short version

Inside one assign call, refer to a newly created column through a lambda, which receives the frame as it stands at that point in the call. Referring to the original frame by name fails, because the new column does not exist on it yet.

The pattern that works

import pandas as pd

df = pd.DataFrame({"a": [1]})

out = df.assign(
    b=lambda d: d["a"] * 2,
    c=lambda d: d["b"] + 1,     # sees b, because d is the updated frame
)
print(out.to_dict("records"))   # [{'a': 1, 'b': 2, 'c': 3}]

The lambda d: in c receives the frame including b, so the dependency works. Keyword arguments are applied in order, which is what makes this possible.

The version that fails

df.assign(
    b=df["a"] * 2,
    c=df["b"] + 1,     # KeyError: 'b' , df is the ORIGINAL frame
)

Here df["b"] is evaluated against the original df, which has no b column, so it raises immediately. The lambda form is what defers the lookup until the column exists.

Why the lambda matters generally

This is the same principle that makes lambdas important throughout a method chain. lambda d: always refers to the frame as it currently is at that step, whereas naming the original variable refers to how it was before the chain started. In a long chain, that difference is the gap between correct and stale.

The alternative: separate statements

You can of course do it in two steps, which is perfectly clear for simple cases.

df["b"] = df["a"] * 2
df["c"] = df["b"] + 1

This works because each line completes before the next begins. The assign form is preferable inside a chain, or when you want to avoid mutating the original frame, but there is nothing wrong with the direct form in a script.

Why prefer assign in a pipeline

assign returns a new frame and leaves the input untouched, which keeps a chain free of side effects and makes each step re-runnable. If you run a cell twice, direct assignment mutates the same frame twice, which can produce surprising results, whereas the chain rebuilds cleanly from the source each time.

Keep the chain readable

If a chain grows past several dependent columns, the lambdas start to obscure the logic. At that point, break it into a named function that takes and returns a frame, and slot it in with pipe. The goal is clarity: a chain that a colleague can follow beats a clever one that they cannot.

Need this solved on your own data?

The library is free and always will be. When the problem is bigger than an answer, we take on the work.

Start a project