A scikit-learn Pipeline connects preprocessing and an estimator as one object. The direct rule for preventing data leakage is: split the observations, place every data-dependent transformation inside the pipeline, and pass the entire pipeline to cross-validation. Never fit imputers, scalers, feature selectors, or category encoders on the complete dataset before validation.

This structure keeps training and inference consistent and supports hyperparameter search. This guide builds on pandas data analysis and machine learning with Python.

What the Pipeline solves

Assume a customer table contains age, income, city, acquisition channel, and a churned target. Numeric values need imputation and scaling; categorical values need imputation and one-hot encoding. Fitting these operations before splitting means that medians, means, and category vocabularies include test-set information. The resulting metric no longer describes truly unseen observations.

A pipeline coordinates fit, transform, and predict. During fit, transformers learn from only the supplied records. During predict, they reuse learned parameters rather than recomputing them. That separation is central to leakage prevention.

Reserve a final test set first

import pandas as pd
from sklearn.model_selection import train_test_split

data = pd.read_parquet('customers.parquet')
X = data.drop(columns='churned')
y = data['churned']


X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.2,
random_state=42,
stratify=y,
)

stratify=y approximately preserves class proportions. It is not a universal choice. If several rows belong to one customer, patient, or device, split by group. If the model predicts future outcomes, split chronologically. Keep the final test set untouched until the pipeline and its decision rules have been selected.

Preprocess heterogeneous columns

ColumnTransformer applies separate workflows to column subsets and concatenates their output. Explicit lists make the input contract reviewable:

from sklearn.compose import ColumnTransformer
from sklearn.impute import SimpleImputer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler

numeric = ['age', 'monthly_income', 'months_active']
categorical = ['city', 'channel']


numeric_steps = Pipeline([
('impute', SimpleImputer(strategy='median')),
('scale', StandardScaler()),
])


categorical_steps = Pipeline([
('impute', SimpleImputer(strategy='most_frequent')),
('encode', OneHotEncoder(handle_unknown='ignore')),
])


preprocess = ColumnTransformer([
('num', numeric_steps, numeric),
('cat', categorical_steps, categorical),
])

handle_unknown='ignore' allows inference when a category was absent from training, encoding that unknown category as zeros in the relevant block. This avoids a runtime error but does not replace drift monitoring. The official ColumnTransformer documentation covers selectors and mixed data.

Attach an estimator

from sklearn.linear_model import LogisticRegression

model = Pipeline([
('preprocess', preprocess),
('classifier', LogisticRegression(
max_iter=1000,
class_weight='balanced',
)),
])


model.fit(X_train, y_train)
probabilities = model.predict_proba(X_test)[:, 1]

The same object accepts original input columns during training and inference. Do not run preprocess.fit_transform(X) first. Likewise, separate pd.get_dummies calls can produce mismatched columns in training and production. The official Pipeline user guide documents chaining and nested parameters.

Run cross-validation correctly

Cross-validation estimates performance variability across several splits. Pass the complete pipeline to cross_validate. For every fold, scikit-learn clones the object and fits imputation, encoding, scaling, and classification only on that fold's training partition:

from sklearn.model_selection import StratifiedKFold, cross_validate

cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
scores = cross_validate(
model,
X_train,
y_train,
cv=cv,
scoring={'roc_auc': 'roc_auc', 'f1': 'f1'},
return_train_score=True,
n_jobs=-1,
)


print(scores['test_roc_auc'].mean())
print(scores['test_roc_auc'].std())

Report both center and spread rather than only the best fold. Compare training and validation metrics to investigate overfitting. Select metrics according to decision costs: ROC AUC does not choose a threshold, while accuracy can obscure failure on a rare class. Use GroupKFold or StratifiedGroupKFold for grouped entities and consider TimeSeriesSplit for ordered observations.

Nested parameter names use double underscores. The search receives the pipeline rather than a matrix transformed ahead of time:

from sklearn.model_selection import GridSearchCV

parameters = {
'preprocessnumimputestrategy': ['mean', 'median'],
'classifierC': [0.1, 1.0, 10.0],
}


search = GridSearchCV(
model,
param_grid=parameters,
scoring='roc_auc',
cv=cv,
n_jobs=-1,
refit=True,
)
search.fit(X_train, y_train)


best_model = search.bestestimator

With refit=True, the best configuration is fitted again on all training data. Evaluate it once on the reserved test set. Repeatedly checking test performance while choosing features, thresholds, or parameters turns the test set into validation and creates an optimistic estimate.

Final evaluation and persistence

from sklearn.metrics import classification_report, roc_auc_score
import joblib

test_probability = best_model.predict_proba(X_test)[:, 1]
test_prediction = best_model.predict(X_test)


print(roc_auc_score(y_test, test_probability))
print(classification_report(y_test, test_prediction, zero_division=0))
joblib.dump(best_model, 'churn_pipeline.joblib')

Persist the complete pipeline because it contains transformations and the estimator. Load artifacts only from trusted sources: pickle-based formats may execute code. Record versions, feature types, target definition, and training period. Apply sustainable Python practices and use pytest to cover missing values, unseen categories, and malformed requests.

Leakage a Pipeline cannot fix alone

  • Post-outcome features: a cancellation timestamp reveals churn even when processed inside the pipeline.
  • Global aggregations: customer averages calculated with future events contaminate historical rows.
  • Duplicate entities: the same person in training and validation can expose individual patterns.
  • Manual test selection: changing decisions after each final metric overfits the test set.
  • Random temporal splits: learning from the future to predict the past gives an unrealistic estimate.

The official chapter on common pitfalls and recommended practices details inconsistent preprocessing and leakage. Correct prevention still requires knowing when each feature becomes available in the real prediction process.

Common errors

  • Fitting StandardScaler, an imputer, or a selector before splitting.
  • Passing only the classifier, rather than the full pipeline, into cross-validation.
  • Including a target-derived or post-outcome column among predictors.
  • Using random K-fold for related groups or chronological records.
  • Tuning and presenting the best score from the same validation as final performance.
  • Saving only the estimator and rebuilding preprocessing separately in the application.

Pre-training checklist

  1. Define the target, prediction time, and information available at that time.
  2. Reserve test data with a strategy appropriate for classes, groups, and time.
  3. Put every learned transformation inside a Pipeline.
  4. Use ColumnTransformer for explicit numeric and categorical contracts.
  5. Cross-validate the full pipeline with decision-relevant metrics.
  6. Tune on training data and evaluate the final test set once.
  7. Persist the complete pipeline and record dependencies and input schema.
  8. Monitor categories, distributions, data quality, and performance after deployment.

Raw features enter one pipeline and predictions leave it. Transformers learn from each training fold without consulting the final test set, and production uses the same object. Temporal and causal reasoning remains essential.