How do I add a row to a DataFrame in pandas?

The short version

Collect your rows in a list and build the DataFrame once at the end. Adding rows one at a time is extremely slow, because each addition copies the whole frame. The DataFrame.append method was removed in pandas 2.0, so use pd.concat when you genuinely need to combine frames.

The right way

import pandas as pd

rows = []
for item in source:
    rows.append({"city": item.city, "amount": item.amount})

df = pd.DataFrame(rows)

Building a list of dictionaries is cheap, and constructing the frame once is a single allocation. This is often hundreds of times faster than growing a frame row by row.

Why growing a frame is so slow

A DataFrame stores its columns in contiguous blocks of memory. Adding a row means allocating new, larger blocks and copying everything across. Doing that once per row means the total work grows with the square of the number of rows, so a loop that feels instant at a hundred rows takes minutes at fifty thousand.

The removed append method

df = df.append({"city": "Leeds"}, ignore_index=True)   # AttributeError in pandas 2.0+

This existed, was slow, and encouraged exactly the anti-pattern above, which is why it was removed. If you find it in old code, replace it, and take the opportunity to restructure into the list-then-construct pattern rather than swapping in a concat inside the same loop.

Combining whole frames

When you really do have several frames to stack, collect them and concatenate once.

frames = [pd.read_csv(p) for p in paths]
df = pd.concat(frames, ignore_index=True)

ignore_index=True renumbers the result, avoiding the repeated index labels that would otherwise follow you around.

Adding a single row to an existing frame

If you genuinely need to add one row to an existing frame (rare in analysis, common in a small script), loc with a new label does it in place.

df.loc[len(df)] = {"city": "York", "amount": 3}

This is acceptable for the occasional row, but do not put it in a loop, for all the reasons above.

The general principle

Any time you find yourself growing a data structure inside a loop, whether it is a DataFrame, a string or a list you keep concatenating, pause. Building the pieces and combining them once is nearly always dramatically faster, and it is usually clearer too, because the construction happens in one obvious place rather than being smeared across an iteration.