一、bagging、
首先bagging算法是集成学习中两大类算法中的其中一个代表算法，还有另一类的经典算法是Xgboost。他们主要的区别是前者学习器之间不存在依赖关系和可以并行生成学习器，后者学习器之间存在强依赖关系和可以串行生成学习器。
bagging算法：（1）bagging算法可以解决回归问题和分类问题。（2）它从原始数据中随机抽取n个样本，重复s次，于是就有个s个训练集，每个训练集都可以训练出一个弱分类器，最终生成s个弱分类器，预测结果将有这些分类器投票决定（选择分类器投票结果中最多的类别作为最后预测结果）。代表的有随机森林。
二、代码实现
import numpy as np
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import BaggingClassifier
from sklearn import datasets
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
X,y = datasets.load_wine(True)
X_train,X_test,y_train,y_test = train_test_split(X,y,random_state = 10244021)
# 一个算法，准确率 71%
knn = KNeighborsClassifier()
knn.fit(X_train,y_train)
knn.score(X_test,y_test)
0.7111111111111111
# 100个算法，集成算法，准确提升到了77.7%
knn = KNeighborsClassifier()
# bag中100个knn算法
bag = BaggingClassifier(base_estimator=knn,n_estimators=100,max_samples=0.8,max_features=0.7)
bag.fit(X_train,y_train)
bag.score(X_test,y_test)
0.7777777777777778
import warnings
warnings.filterwarnings('ignore')
lr = LogisticRegression()
lr.fit(X_train,y_train)
lr.score(X_test,y_test)
0.9555555555555556
bag = BaggingClassifier(base_estimator=LogisticRegression(),n_estimators=100,
max_samples=0.7,max_features=0.5)
bag.fit(X_train,y_train)
bag.score(X_test,y_test)
0.9555555555555556
clf = DecisionTreeClassifier()
clf.fit(X_train,y_train)
clf.score(X_test,y_test)
0.9333333333333333
bag = BaggingClassifier(base_estimator=DecisionTreeClassifier(),n_estimators=100,
max_samples=1.0,max_features=0.5)
bag.fit(X_train,y_train)
bag.score(X_test,y_test)
0.9777777777777777
import matplotlib.pyplot as plt
from sklearn import tree
plt.figure(figsize=(9,9))
_ = tree.plot_tree(bag[0],filled=True)
output_10_0.png
plt.figure(figsize=(9,9))
_ = tree.plot_tree(bag[1],filled=True)
output_11_0.png
X_train.shape
(133, 13)
133*0.7
93.1
32+36+25
93
plt.figure(figsize=(9,9))
_ = tree.plot_tree(bag[2],filled=True)
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
<ipython-input-1-b56635e6beeb> in <module>
----> 1 plt.figure(figsize=(9,9))
2 _ = tree.plot_tree(bag[2],filled=True)
NameError: name 'plt' is not defined
pd.describe()
