How do I apply different preprocessing to numeric and categorical columns in scikit-learn?

The short version

Use ColumnTransformer to send each group of columns to the right preprocessing: numeric columns to a scaler, categorical columns to a one-hot encoder. It applies each transformer to its columns and joins the results. Wrap it in a Pipeline with your model so everything is fitted on the training data only and applied consistently to new data.

Building the transformer

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import StandardScaler, OneHotEncoder

numeric = ["length", "width"]
categorical = ["colour"]

preprocess = ColumnTransformer([
    ("num", StandardScaler(), numeric),
    ("cat", OneHotEncoder(), categorical),
])

Each tuple is a name, a transformer, and the columns it applies to. ColumnTransformer runs them in parallel and concatenates the outputs into one feature matrix.

Why you need it

Numeric and categorical columns need different treatment: numbers should be scaled, categories should be encoded into indicator columns. Applying a scaler to a text column fails, and one-hot encoding a numeric column is meaningless. ColumnTransformer lets you do the right thing to each type in a single, tidy step, instead of transforming subsets by hand and stitching them back together.

Putting it in a pipeline

Combine it with the model so the whole workflow is one leak-proof object.

from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression

model = Pipeline([
    ("prep", preprocess),
    ("clf", LogisticRegression()),
])
model.fit(X_train, y_train)

Now fit learns the scaling and the category set from the training data only, and predict applies the same transformations to new data, which is exactly what avoids leakage and mismatches.

Handling unseen categories

If new data contains a category the encoder did not see during fitting, the default raises an error. For data where new categories are possible, tell the encoder to ignore them rather than fail.

OneHotEncoder(handle_unknown="ignore")

This encodes an unseen category as all zeros instead of crashing, which is usually the safer choice for a model that will meet fresh data.

Keeping the untouched columns

By default, columns not listed are dropped. To keep them as they are, set remainder="passthrough".

ColumnTransformer([...], remainder="passthrough")

Use this when some columns are already in the right form and should flow through unchanged, rather than being silently discarded.

Why this is the standard structure

A ColumnTransformer inside a Pipeline is the backbone of a real tabular model: it documents which columns get which treatment, fits everything on training data only, applies it identically at prediction, and can be tuned and saved as a single unit. Learning this pattern removes most of the fiddly, error-prone preprocessing glue people otherwise write by hand.