基于sklearn建立机器学习的pipeline

Scikit-learn Pipeline可以简化机器学习代码，让我们的代码看起来更加条理。

构建pipeline的流程如下例子：

from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.impute import SimpleImputer
from sklearn.preprocessing import OneHotEncoder
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_absolute_error

# 找到分类变量
categorical_cols = [cname for cname in X_train_full.columns if
                    X_train_full[cname].nunique() < 10 and 
                    X_train_full[cname].dtype == "object"]

# 找到数值变量
numerical_cols = [cname for cname in X_train_full.columns if 
                X_train_full[cname].dtype in ['int64', 'float64']]

# 缺失值填补
numerical_transformer = SimpleImputer(strategy='constant') 

# 对分类变量的处理
categorical_transformer = Pipeline(steps = [
    ('imputer', SimpleImputer(strategy = 'most_frequent')),
    ('onehot', OneHotEncoder(handle_unknown = 'ignore'))]) 

# Bundle preprocessing for numerical and categorical data
preprocessor = ColumnTransformer(
    transformers=[
        ('num', numerical_transformer, numerical_cols),
        ('cat', categorical_transformer, categorical_cols)
    ])

# Define model
model = RandomForestRegressor()

# Bundle preprocessing and modeling code in a pipeline
clf = Pipeline(steps=[('preprocessor', preprocessor),
                      ('model', model)
                     ])

# Preprocessing of training data, fit model 
clf.fit(X_train, y_train)

# Preprocessing of validation data, get predictions
preds = clf.predict(X_valid)

print('MAE:', mean_absolute_error(y_valid, preds))

简单来说主要流程就是：
1). 对分类变量和数值变量分别进行缺失值处理；
2). 对数值变量编码 & 对分类变量标准化（scale）；
3). 建立机器学习模型；
4). 将其合到一起组成pipeline;
5). 预测

以上学习自：https://www.kaggle.com/alexisbcook/pipelines